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ABSTRACT 


This  research  was  performed  to  determine  the  factors  which  affect  the  quality  of  situation 
awareness  (S  A)  in  civilian  aircrew  members.  The  recent  popularity  and  increased 
investigation  of  SA  has  forced  the  research  community  to  be  very  specific  in  modeling  how 
it  enhances  decision-making  in  complex,  dynamic  environments.  This  research  sought  to 
better  understand  the  effects  of  poor  conununication,  high  workload  and  time  pressure  on 
losses  of  SA  for  teams  and  individuals.  Based  upon  Endsley’s  (1995a)  model  of  SA  and 
an  analysis  of  aviation  incident  reports,  it  was  found  that  205  of  the  590  incident  reports 
investigated  involved  a  loss  in  SA.  This  implies  that  SA  is  indeed  a  significant  factor 
affecting  aviation  decision-making.  Further,  the  results  show  that  instances  of  poor 
communication  and  high  workload  occur  in  significantly  higher  proportions  of  team 
comprehension  errors.  These  findings  point  to  both  the  usefulness  of  Endsley’s  (1995a) 
model  and  the  usefulness  of  using  incident  report  data  for  the  understanding  of  SA. 
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CHAPTER  I 


INTRODUCTION 

A  significant  amount  of  recent  research  has  assessed  the  situation 
awareness  (SA)  of  people  who  are  involved  in  work  environments  which  are 
considered  "dynamic,  complex,  high  information  loaded,  variably  workloaded, 
and  risky"  (Gaba,  Howard,  and  Small,  1995,  p.  20).  Specifically,  researchers  have 
been  trying  to  determine  the  aspects  of  the  construct  and  ways  in  which  to  best 
train  individuals  to  improve  their  situation  awareness.  This  construct,  which  can 
be  viewed  as  a  state,  known  as  SA,  has  taken  the  both  the  military  and  civilian 
research  communities  by  storm  (Dominguez,  1994).  A  somewhat  unified 
definition  of  SA  has  emerged  in  the  literature.  SA  is  "the  perception  of  the 
elements  in  the  environment  within  a  volume  of  time  and  space,  the 
comprehension  of  their  meaning,  and  the  projection  of  their  status  in  the  near 
future"  (Endsley,  1995a,  p.  36).  SA  is  a  product  or  cognitive  state  that  exists  after 
one  has  assessed  the  environment  and  the  information  it  affords  at  a  particular 
point  in  time. 

This  definition  is  certainly  not  the  only  definition  which  has  been 
presented  addressing  the  construct  of  SA.  Other  researchers  from  various 
backgrounds  have  provided  their  own  definitions  and  subsequent  theories  to 
describe  this  construct  (Dominguez,  1994).  Researchers  have  defined  SA  as 
"externally  directed  consciousness  "  (Smith  and  Hancock,  1995),  "a  label 
describing  a  number  of  cognitive  processes  that  are  critical  to  dynamic,  event- 
driven,  and  multitask  fields  of  practice"  (Sarter  and  Woods,  1995),  or  even  "a 
pilot's  continuous  perception  of  self  and  aircraft  in  relation  to  the  dynamic 
environment  of  flight,  threats,  and  mission,  and  the  capability  to  forecast,  and 
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execute  tasks  based  on  that  perception"  (Vidulich,  1994).  As  one  can  see,  these 
definitions  while  different,  do  share  some  theoretical  underpinnings. 

Within  the  context  of  this  research,  the  author  has  chosen  to  focus  on  the 
model  presented  by  Endsley  (1995a).  Endsley's  model  is  the  most  theoretically 
developed  and  widely  tested  to  date.  Further  research  of  her  model  may  be  most 
beneficial  for  the  sake  of  advancement  of  theory  and  knowledge  concerning  the 
construct  of  SA. 


Situation  Awareness 

One  may  wonder,  in  what  circumstances  may  SA  be  critical?  Consider  the 
following  real  world  example  of  team  SA: 

"Date:  December  29, 1972 
Type:  Lockheed  L-1011 
Registration:  N310EA 
Operator:  Eastern  Air  Lines 
Where:  Miami,  FL 
Report  No.  NTSB- AAR-73-14 
Report  Date:  June  14, 1973 
Pages:  45 

An  Eastern  Air  Lines  Lockheed  L-1011  crashed  at  2342  eastern  standard 
time,  December  29, 1972, 18.  7  miles  west-northwest  of  Miami 
International  Airport,  Miami,  Florida.  The  aircraft  was  destroyed.  Of 
the  163  passengers  and  13  crewmembers  aboard,  94  passengers  and  5 
crewmembers  received  fatal  injuries.  Two  survivors  died  later  as  a  result 
of  their  injuries. 

Following  a  missed  approach  because  of  a  suspected  nose  gear  malfunction, 
the  aircraft  climbed  to  2,000  feet  mean  sea  level  and  proceeded  on  a 
westerly  heading.  The  three  flight  crewmembers  and  a  jumpseat  occupant 
became  engrossed  in  the  malfunction. 

The  National  Transportation  Safety  Board  determined  that  the  probable 
cause  of  this  accident  was  the  failure  of  the  flightcrew  to  monitor  the 
flight  instruments  during  the  final  4  minutes  of  flight,  and  to  detect  an 
unexpected  descent  soon  enough  to  prevent  impact  with  the  ground. 
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Preoccupation  with  a  malfunction  of  the  nose  landing  gear  position 
indicating  system  distracted  the  crew's  attention  from  the  instruments 
and  allowed  the  descent  to  go  unnoticed. 

As  a  result  of  the  investigation  of  this  accident,  the  Safety  Board  has 
made  recommendations  to  the  Administrator  of  the  Federal  Aviation 
Administration."  (NTSB,  1973). 

From  this  report,  one  can  note  that  the  perception  of  information  in  the 
environment,  the  comprehension  of  what  that  information  means,  and  the 
projection  of  future  possible  status  of  an  aircraft  is  extremely  important  and, 
apparently,  sometimes  difficult. 

Accidents  such  as  these  have  been  the  driving  force  behind  the  study  of 
SA.  Vidulich  (1994)  has  pointed  out  that  it  is  not  just  other  researchers  who 
would  like  to  understand  this  construct,  but  also  aviation  pilots.  Pilots  were  a 
major  impetus  behind  SA  research  because  traditional  human  information 
processing  research  has  not  provided  pilots  the  answers  and  training 
interventions  needed  to  deal  with  the  difficulties  of  working  in  dynamic 
environments  (Vidulich,  1994).  The  current  question  is  no  longer  whether  SA 
exists,  but  rather  what  are  the  components  of  SA,  the  factors  which  may  effect  it, 
how  should  SA  be  measured,  and  how  can  it  be  improved  through  training. 

Most  of  the  current  research  describing  SA  has  centered  around  pilots, 
specifically  US  Air  Force  and  US  Navy  jet  pilots  (Endsley,  1995a).  The  current 
database  of  information  purports  that  "in  the  flight  environment,  the  safe 
operation  of  the  aircraft  in  a  manner  consistent  with  the  pilot's  goals  is  highly 


4 


dependent  on  a  current  assessment  of  the  changing  situation,. ...aircraft 
operational  parameters,  external  conditions,  navigational  information,  other 
aircraft  and  hostile  factors"  (Endsley,  1995a,  p.  49).  Thus,  if  the  pilot  or 
individual  in  a  complex  environment  is  not  totally  aware  of  all  of  the  factors 
around  him  or  her,  performance  may  decline  and  the  goals  of  work  may  not  be 
achieved. 


Endsley' s  Theory  of  Situation  Awareness 
Researchers  have  begun  to  develop  the  theory  behind  SA  and  the  factors 
which  may  have  an  effect  on  SA  in  dynamic  and  complex  environments.  Endsley 
(1995a)  offers  a  comprehensive  theory  of  SA  including  the  role  of  environmental 
and  cognitive  factors.  This  model  is  shown  as  Figure  1. 


Figure  1.  Endsley's  (1995a)  Model  of  Situation  Awareness. 
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Her  model  presents  a  number  of  factors  which  can  have  an  effect  upon  one's 
ability  to  maintain  SA  including  the  perception  and  comprehension  of  the 
current  environment,  and  projection  of  future  status  of  the  activities  in  a  dynamic 
environment. 

As  one  can  see  from  Figure  1,  Endsley's  model  corresponds  to  the  human- 
machine  interface  model  depicted  in  most  introductory  human  factors 
psychology  books  (Proctor  and  Van  Zandt,  1994;  R.  Nowaczyk,  personal 
communication,  September  19, 1996).  She  shows  that  both  the  environment, 
being  the  Task/System  Factors,  and  the  Individual  Factors  will  both  have  an 
impact  upon  one's  or  a  team's  SA  at  the  interface  of  the  two.  The  external 
environmental  factors  which  will  have  an  effect  on  SA  are  workload,  numerous 
forms  of  stress  (time  pressure,  company  pressure,  etc.),  interface  design,  task 
complexity,  and  automation.  The  internal  factors  which  will  have  an  effect  upon 
SA  will  be  experience,  communication,  pre-attentive  processing,  attention, 
working  memory,  perception,  long-term  memory,  automaticity,  one's  goals,  and 
mental  models  (Endsley,  1995a).  While  this  list  of  factors  may  seem  to  be 
somewhat  of  a  laundry  list  of  possible  factors  affecting  SA,  one  must  remember 
that  this  construct  is  the  combination  of  three  very  important  cognitive  processes 
working  in  conjunction  with  each  other,  those  being  perception  and 
comprehension  of  information,  and  projection  of  future  status.  By  examining 
how  SA  might  work  in  a  specific  situation,  one  can  better  understand  how  these 
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factors  might  affect  SA.  The  following  is  an  example  of  how  SA  would  be 

utilized  to  enhance  decision-making  while  flying  an  advanced  jet  aircraft: 

"A  pilot  may  perceive  several  aircraft  recognized  as  enemy  fighter  jets 
that  are  approaching  in  a  particular  spatial  arrangement.  By  pattern 
matching  to  prototypes  in  memory,  these  separate  pieces  of 
information  can  be  classified  as  a  particular  recognized  aircraft 
formation.  According  to  an  internally  held  mental  model,  the  pilot  is 
able  to  generate  probable  attack  scenarios  for  this  type  of  formation 
when  in  relation  to  an  aircraft  with  the  location  and  flight  vector  of  his 
or  her  ownship.  Based  on  this  high  level  of  SA,  the  pilot  is  then  able  to 
select  the  prescribed  tactics  that  dictate  exactly  what  evasive  maneuvers 
should  be  taken."  (Endsley,  1995a,  p.  44.) 


While  this  example  is  geared  toward  those  in  the  fighter  aircraft 
community,  one  can  see  that  SA  has  three  distinct  levels:  perception  (seeing  the 
aircraft),  comprehension  (recognizing  those  aircraft  as  enemy  fighters),  and 
projection  of  future  status  (generations  of  attack  possibilities  ultimately  linked 
with  performance).  In  the  following  sections,  these  factors  are  examined  in 
greater  detail. 

Factors  Proposed  to  Affect  Situation  Awareness 
Internal  Factors 

Attention 

As  Endsley  states,  "in  complex  and  dynamic  environments,  attention 
demands  resulting  from  information  overload,  complex  decision-making,  and 
multiple  tasks  can  quickly  exceed  a  person's  limited  attention  capacity"  (1995a, 
p.  41).  Objects  in  a  dynamic  environment  will  largely  effect  one's  momentary 
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SA.  While  the  number  of  tasks  which  may  be  completed  successfully 
simultaneously  varies,  previous  research  has  shown  that,  in  general,  as  the 
number  of  extra  activities  a  human  operator  is  forced  to  accomplish 
simultaneously  increases,  performance  on  a  primary  task  is  adversely  affected 
(Proctor  and  Van  Zandt,  1994).  Specifically,  it  has  been  found  that  multiple 
sources  of  data  can  be  monitored  by  a  human  operator  in  a  comparatively 
efficient  manner.  However,  one's  ability  to  monitor  multiple  sources  of  data 
declines  as  information  is  received  simultaneously  from  numerous  sources 
(Proctor  and  Van  Zandt,  1994). 

One  of  the  most  relevant  theories  of  attention  for  the  discussion  of  SA  is 
the  multiple  resource  theory  of  attention  by  Wickens  (1992).  This  theory  "argues 
that  instead  of  one  single  supply  of  undifferentiated  resources,  people  have 
several  different  capacities  with  resource  properties"  (Wickens,  1992,  p.  375). 
Different  modalities  of  input  (visual  and  auditory)  and  different  processing 
codes  (spatial  and  verbal)  exist  to  allow  the  human  to  complete  multiple  tasks 
successfully  and  attend  to  different  pieces  of  information  as  long  as  the  tasks  use 
differing  resources.  As  the  demand  of  resources  increases,  performance 
decreases  on  complex  tasks.  This  theory  explains  to  an  adequate  degree  how 
time-sharing  between  different  attentional  demands  can  occur  without  difficulty 
given  adequate  time  and  resource  allocation.  Attentional  demands  may  be 
somewhat  high  for  SA  if  the  human  must  allocate  the  same  resources  to  attend  to 
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numerous  pieces  of  information  or  process  them  within  the  same  or  competing 
processing  codes  (Endsley,  1995a). 

Working  Memory 

While  attention  is  crucial  for  SA,  human  working  memory  capacity  may 
be  the  most  important  factor  affecting  SA.  Recent  work  by  Baddeley  (1995)  has 
shown  both  the  capacity  and  limitations  of  human  working  memory.  Baddeley's 
model  of  working  memory  postulates  that  a  central  executive  exists  for 
attentional  control  which  is  supported  by  two  storage  systems,  the  visuo-spatial 
sketchpad  and  a  phonological  loop.  The  central  executive  is  responsible  for 
strategy  selection  and  planning  (Baddeley,  1995).  This  component  of  working 
memory  is  thought  to  be  the  link  between  the  two  different  storage  systems.  The 
visuospatial  sketchpad  is  a  storage  system  responsible  for  holding  nonverbal 
spatial  information  in  working  memory.  The  phonological  loop  would  be  the 
sketchpad's  counterpart,  responsible  for  working  storage  of  verbal  information. 
Because  humans  have  only  a  limited  working  memory  capacity  (Anderson,  1995; 
Baddeley,  1995;  Endsley,  1995a)  only  a  limited  number  of  mental  tasks  can  be 
accomplished  at  one  specific  time.  Working  memory  is  relevant  for  SA  as  it  may 
prove  to  be  a  limitation  for  SA  as  spatial  or  verbal  information  may  overload 
working  memory  storage  capacity  as  well  as  the  central  executive,  which  may 


lead  to  poor  SA. 
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Long-Term  Memory  and  Experience 

Long-term  memory  and  experience  are  hypothesized  to  also  influence  SA. 
Experience  has  long  be  contended  as  a  significant  predictor  of  performance  in 
many  contexts.  Within  the  realm  of  cognitive  psychology,  experience  and 
expertise  have  been  the  subject  of  numerous  studies  (Anderson,  1995). 
Researchers  have  determined  that  as  practice  occurs  within  problem  solving, 
skill-based  production,  or  physical  activities  such  as  sports,  a  power  law  of 
learning  exists  (Anderson,  1995).  This  law  suggests  that  as  people  practice  their 
actions,  they  learn  at  exponential  rates  until  further  practice  has  no  real  benefit. 

One  might  wonder  how  experience  may  effect  SA.  It  has  been 
hypothesized  that  as  pilots  or  people  working  in  other  dynamic  environments 
complete  their  daily  duties  routinely,  they  store  and  retain  the  problems  they 
face  within  their  daily  jobs  in  long-term  memory.  This  effect  is  synonymous  with 
the  study  of  expert  chess  players  who  do  not  remember  the  moves  for  individual 
chess  pieces,  but  rather  a  pattern  of  moves  for  several  pieces  (Anderson,  1995). 

Studies  have  shown  that  as  players  increase  their  knowledge  of  the  game 
over  long  periods  of  time  and  build  on  their  experience,  they  store  these  patterns 
of  moves  in  long-term  memory  allowing  for  a  reduction  in  errors  while  playing 
and  for  better  strategic  planning  capabilities  (Anderson,  1995).  Schema 
formation  occurs.  An  application  of  this  concept  for  SA  would  suggest  that  if  a 
pilot  is  inexperienced  with  an  aircraft,  the  copilot,  a  certain  approach,  or  any 
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number  of  distracters  on  the  flight  deck,  SA  will  suffer  due  to  a  lack  of 
information  (patterris,  scripts,  mental  models,  schemata)  stored  in  long-term 
memory.  One  will  have  to  focus  more  attention  on  solving  individual  problems 
continuously  which  could  easily  lead  one  to  become  "behind  the  plane"  or  have 
poor  quality  SA.  Having  long-term  storage  of  information  relevant  to  a  dynamic 
and  rapidly  changing  environment  should  definitely  improve  SA. 

Automaticity 

Automaticity  is  also  an  important  contributor  to  the  experience/ expertise 
factor.  Again,  cognitive  researchers  have  shown  that  as  actions  are  continually 
practiced  they  can  become  automatic,  that  is,  requiring  no  conscious  attentional 
resources.  One  of  the  seminal  experiments  of  this  phenomenon  was  conducted 
by  Schneider  and  Shiffrin  (1977).  Subjects  were  required  to  scan  visual  displays 
for  a  certain  target  (letter).  With  vast  amounts  of  practice,  the  experimenters 
found  that  the  experimental  task  became  automatic  for  the  subjects  in  that  task 
completion  did  not  require  conscious  effort  or  attentional  resources  (Schneider 
and  Shiffrin,  1977). 

This  same  phenomenon  is  experienced  by  most  drivers  everyday.  One 
can  often  remember  occurrences  where  one  appears  inattentive  during  the  drive 
home.  Even  complex  tasks  such  as  driving  a  car  or  piloting  an  aircraft  can  be 
automatized  to  some  degree.  According  to  the  theory  of  SA,  one  who  completes 
a  drive  in  this  manner  is  still  maintaining  SA,  perhaps  unconsciously.  It  has  been 
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shown,  for  instance,  that  feature  detection  and  synthesis  (an  automatic  process) 
occur  as  a  function  of  how  familiar  those  patterns  are  to  a  person  (LaBerge,  1973). 
When  unexpected  stimuli  are  presented,  the  driver  shifts  from  automatic  to 
controlled  processing  which  uses  working  memory  and  attentional  resources. 
Thus,  SA  may  be  influenced  by  both  automatic  and  controlled  (conscious  and 
effortful)  processing. 

Endsley's  theory  of  SA  proposes  that  even  complex  tasks  such  as  driving 
may  not  provide  adequate  salience  to  be  recalled  if  only  familiar  stimuli  were 
encountered  on  a  drive  home  (1995a).  However,  as  automaticity  may  lessen  the 
attentional  demands  and  increase  the  chances  that  SA  will  be  maintained,  one 
must  consider  how  unfamiliar  stimuli  such  as  a  hysterical  passenger  on  an 
aircraft  may  affect  a  pilot's  or  crew's  SA,  or  how  the  unexpected  rise  and  fall  of 
two  distinct  physiological  parameters  may  affect  an  anesthesiologist's  SA. 
Research  conducted  by  LaBerge  (1973)  has  shown  that  with  unfamiliar  stimuli, 
the  attention  shift  to  an  unexpected  pattern  requires  time  and  controlled 
attentional  resources.  However,  with  significant  practice,  one  can  complete  this 
shift  to  recognize  new  patterns  when  primed.  However,  this  research  was 
conducted  in  a  static  environment  (laboratory  with  simple  pattern  recognition 
tasks).  Research  in  dynamic  environments  is  needed  to  determine  if  automatic 
processes  will  be  affected  by  changing  conditions. 
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Environmental  Factors 

Workload 

According  to  Endsley's  theory,  the  main  moderating  variable  for  SA  will 
be  one's  workload,  especially  one's  mental  workload  (1995a).  While  Endsley 
(1993)  has  shown  that  the  constructs  of  SA  and  workload  can  diverge,  the 
assumption  remains  that  as  workload  increases,  the  operator  must  expend  effort 
and  attention  to  achieve  SA.  Therefore,  SA  is  likely  to  suffer. 

A  prime  example  of  how  workload  should  affect  SA  is  seen  in  one  of  the 
most  prominent  findings  of  psychology,  the  Yerkes-Dodson  Law  (Yerkes  and 
Dodson,  1908).  The  main  findings  of  this  law  show  that  as  arousal  increases, 
complex  task  performance  first  increases,  but  then  rapidly  decreases.  Because  SA 
is  to  be  maintained  in  a  complex  and  dynamic  environment,  as  arousal  level 
increases,  one's  ability  to  project  the  future  status  of  the  dynamics  of  a  situation 
should  suffer.  This  finding  has  been  attributed  to  the  relationship  between 
workload  and  attention.  Researchers  have  shown  that  as  arousal  increases,  a 
phenomenon  known  as  perceptual  narrowing  occurs  which  causes  one's 
attention  to  become  more  focused  (Proctor  and  Van  Zandt,  1994).  "The  ability  to 

discriminate  between  relevant  and  irrelevant  cues  decreases . at  high  arousal 

levels  the  allocation  of  attention  is  controlled  by  fewer  and  often  less  appropriate 
features  of  the  situation"  (Proctor  and  Van  Zandt,  1994,  p.  196-197). 
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The  Team  and  Situation  Awareness 

Researchers  have  explored  individual  SA,  however,  team  SA  is  thought  to 
be  a  much  more  complex  entity  than  individual  SA  simply  because  of  the  added 
team  element  (Salas,  Prince,  Baker,  and  Shreshtha,  1995).  To  understand  this 
phenomenon  of  team  SA,  several  researchers  have  investigated  both  the 
literature  concerning  teams,  and  also  that  of  SA  in  order  to  develop  a  strong  and 
guiding  definition  of  this  construct  for  further  study. 

These  researchers  first  describe  the  concept  of  a  team  in  order  to  lead  to 
the  development  of  team  SA.  The  team  is  defined  as  "a  set  of  two  or  more 
people  who  interact,  dynamically,  interdependently,  and  adaptively  toward  a 
common  and  valued  goal/ objective/ mission,  who  have  each  been  assigned 
specific  roles  or  functions  to  perform,  and  who  have  a  limited  life-span  of 
membership"  (Salas,  Dickinson,  Converse,  and  Tannenbaum,  1992).  Working 
with  this  constraint  of  "team,"  the  definition  of  team  SA  can  be  described  as  a 
"dynamically  changing  state  affected  by  the  process  of  situation  assessment  and 
other  team  process  behaviors,  the  level  of  each  members  own  individual  SA,  and 
most  importantly  the  level  of  shared  understanding  among  team  members" 
(Stout,  Cannon-Bowers,  and  Salas,  in  press).  Within  this  definition  of  team  SA 
lies  a  very  important  piece  of  information.  In  order  to  truly  have  team  SA,  a 
team  must  have  a  shared  understanding  of  the  environment,  a  shared 
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understanding  of  the  factors  affecting  that  environment,  and  a  shared  future 
projection  of  how  to  deal  with  the  factors  in  that  environment  (Endsley,  1995b). 

Do  teams  that  have  a  higher  level  of  team  SA  out  perform  those  teams 
which  do  not  have  a  high  degree  of  SA?  While  the  research  in  this  area  is  very 
recent,  it  seems  to  indicate  that  the  more  SA  a  team  possesses,  the  better  the  team 
will  perform.  Specifically,  research  has  shown  that  aviation  crews  which  have  a 
higher  degree  of  communication  and  situation  assessment  (constructs  linked  to 
team  SA)  have  better  performance  (Orasanu,  1995a). 

Orasanu  reviewed  the  utterances  of  high-performing  and  low-performing 
aircrews  when  they  were  faced  with  high  levels  of  information  overload 
emergency  situations  in  a  flight  simulator.  This  research  found  high-performing 
crews  performed  more  information  gathering  and  planning  when  faced  with 
dynamic  problem  conditions  than  low-performing  crews.  It  is  assumed  from 
these  data  that  the  high-performing  crews  understood  the  situation  surrounding 
them  to  a  higher  degree  than  low-performing  crews.  The  high-performing  crews 
"engaged  in  more  planning...  which  essentially  reflected  their  SA  to  prepare  for 
what  may  develop  or  for  high  workload  episodes"  (Orasanu,  1995a,  p.5).  While 
this  finding  seems  extremely  plausible,  the  samples  were  limited  in  number 
restricting  the  external  validity  of  this  result.  This  research,  however,  shows  the 
potential  importance  of  team  SA  in  team  situations. 
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A  recent  review  of  the  literature  on  team  SA  by  Stout  et  al.,  (in  press) 
examined  studies  of  team  SA  and  performance  in  dynamic  settings.  The 
researchers  reviewed  seven  articles  dealing  with  team  SA  and  found  that  in  each 
case,  team  SA  was  significantly  related  to  team  performance.  It  is  important  to 
note  that  specific  team  process  behaviors  moderated  the  relationship  between 
team  SA  and  performance.  These  team  process  behaviors  included  active 
communication,  information  gathering  after  critical  decisions  had  been  made  in 
the  team  task,  and  pre-planning  task  objectives  (pre-briefing)  (Stout  et  al.,  in 
press).  It  seems  as  though  team  SA  is  a  crucial  construct  that  must  be  maintained 
in  order  for  teams  to  perform  well. 

Time  Pressure 

As  with  workload,  time  pressure  is  hypothesized  to  be  a  stressor  which 
should  have  a  detrimental  effect  upon  SA.  Endsley's  model  assumes  that  under 
the  stress  of  time  pressure,  operators  may  be  forced  to  adopt  a  narrower  span  of 
attention,  or  to  make  decisions  in  a  premature  marmer  where  a  failure  to  explore 
an  integrate  all  relevant  information  has  occurred.  Endsley  (1995a)  goes  on  to 
say  that  this  narrowing  of  attention  will  significantly  negatively  affect  the 
perception  of  information. 

Recent  studies  into  the  effects  of  stress  on  team  and  individual  decision 
making  have  found  that  under  the  stress  of  time  pressure  (as  well  as  other  forms 
of  stress),  teams  experience  a  loss  in  team  perspective  as  well  as  a  decrement  in 
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task  performance  (Driskell,  Johnston,  and  Salas,  1994).  What  this  suggests  for  SA 
is  that  in  a  team  environment,  as  stress  increases  through  time  pressure,  a 
narrowing  of  team  perspective  or  decrease  in  shared  understanding  of  the  task 
can  occur.  Thus,  it  can  be  seen  that  stress  factors  such  as  time  pressure  should 
have  varying  effects  upon  both  individual  and  team  SA. 

Errors  in  Situation  Awareness 

Within  Endsley's  model,  SA  is  described  in  terms  of  levels  or  components. 
The  levels  are  (1)  perception  of  the  environmental  invariants  in  a  volume  of 
space  and  time,  (2)  the  comprehension  of  that  information  for  meaningfulness, 
and  (3)  the  projection  of  near  future  status  based  on  information  presented 
(Endsley,  1995a).  These  three  levels  of  SA  are  important  because  each  level  can 
and  should  have  specific  types  of  errors  associated  with  it.  Endsley  (1995a)  lays 
out  the  types  of  errors  which  may  occur  in  each  of  these  levels. 

Within  Level  1  of  SA,  operators  must  perceive  the  information  around 
them  in  order  to  have  higher  quality  SA.  Thus,  the  main  error  which  would 
occur  in  this  level  would  be  missing  information  which  is  needed  to  maintain  SA 
(Endsley,  1995a).  The  Eastern  Air  Lines  L-1011  crash  mentioned  earlier 
demonstrates  how  a  lack  of  information  can  lead  to  poor  quality  Level  1  SA  and 
poor  performance.  As  the  crew  focused  its  attention  on  the  landing  gear 
indicator  light,  no  information  was  gathered  concerning  the  slow  descent  of  the 
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plane.  This  narrowing  of  attention  among  the  crewmembers  led  to  the  poor 
quality  SA  and  subsequent  disaster. 

Level  2  SA  is  characterized  as  the  comprehension  of  information  which  is 
acquired  during  the  previous  level  of  SA.  Endsley  (1995a)  states  that  most 
instances  of  poor  SA  will  be  at  Level  2  and  occur  as  a  result  of  "inability  to 
properly  integrate  or  comprehend  the  meaning  of  perceived  data  in  light  of 
operator  goals"  (p.  55).  She  goes  on  to  state  that  this  inability  to  properly 
integrate  information  could  be  due  to  inexperience  with  a  situation  where  the 
operator  lacks  the  scripts  and  schemata  in  long-term  storage  to  effectively  deal 
with  the  information,  or  a  person  will  select  the  wrong  mental  model  from 
memory  to  interpret  the  available  information,  thus  leading  to  misinterpretation 
of  information. 

In  a  recent  review  of  the  37  fatal  accidents  involving  US  airline  carriers, 
the  National  Transportation  Safety  Board  (NTSB)  determined  that  51  of  the  302 
classified  errors  which  occurred  during  these  flights  were  attributable  to  poor 
tactical  decision-making  (NTSB,  1994).  Here,  tactical  decision-making  was 
classified  as  "failing  to  change  a  course  of  action  in  response  to  signal  to  do  so; 
failing  to  heed  warnings  or  alerts  that  suggest  a  change  in  course  of  action" 
(NTSB,  1994,  p.  47).  According  to  theory,  these  error  types  would  most  likely 
have  occurred  as  the  crew  members  misinterpreted  perceived  information,  a 


Level  2  SA  error. 
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The  third  level  of  SA  is  characterized  as  the  projection  of  near  future 
status  based  on  the  information  presented  to  an  operator.  This  information 
should  help  an  operator  make  better  and  more  efficient  decisions  in  the  face  of  a 
changing  environment.  Endsley  (1995a)  theorizes  that  errors  affecting  this  level 
of  SA  would  occur  because  the  operators  do  not  have  a  highly  developed  mental 
model  of  a  situation,  or  they  are  simply  not  proficient  at  mental  simulation. 
According  to  a  list  of  possible  actions  which  may  occur  as  a  result  of  having 
Level  3  SA,  Orasanu  (personal  communication,  June  25, 1996)  states  that 
crewmembers  would  prioritize  and  assign  duties  to  others,  alert  or  warn  their 
fellow  crewmembers,  or  make  contingency  plans  based  on  the  information  they 
had  interpreted. 

This  taxonomy  of  actions  seems  to  have  some  validity  as  the  most 
frequent  error  made  by  crewmembers  in  fatal  accidents  from  1978-1990  were 
resource  management  errors  (NTSB,  1994).  Resource  management  errors  are 
characterized  as  "failing  to  assign  task  responsibilities  or  distribute  tasks  among 
crewmembers;  failing  to  prioritize  task  accomplishment;  overloading 
crewmembers;  failing  to  transfer/ assume  control  of  the  aircraft"  (NTSB,  1994,  p. 
47).  According  to  Endsley's  theory,  these  could  be  classified  as  Level  3  SA  errors 
as  the  projection  of  future  status  would  require  operators  to  manage  cockpit 
resources  which  clearly  did  not  occur  according  to  the  NTSB. 
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While  these  types  of  SA  errors  may  occur  within  dynamic  environments, 
the  operator  in  the  loop  may  never  suffer  total  loss  of  SA.  However,  one  can 
determine  whether  one's  or  the  team's  SA  is  of  high  or  low  quality.  SA  of  high 
quality  would  exist  as  the  operational  requirements  of  the  definition  of  SA  are 
maintained,  i.e.,  perception,  comprehension  and  projection  of  future  dynamics. 
Similarly,  low  quality  SA  would  exist  when  an  error  has  occurred  in  any  level  of 
SA. 


The  Present  Study 

The  purpose  of  this  study  is  to  test  Endsley's  model  of  SA  through  the 
analysis  of  incident  report  data.  SA  has  been  studied  using  a  number  of  methods 
such  as  process  indices  (verbal  protocols,  process  tracing),  subjective  assessment 
of  SA  (numerous  scales  exist),  questionnaires  (Endsley's  Situation  Awareness 
Global  Assessment  Technique),  behavioral  measures  (i.e.,  communication 
frequency),  and  performance  measures  (Endsley,  1996).  While  these  techniques 
exist,  each  only  measures  a  portion  of  SA.  Also,  these  differing  methods  of  study 
also  vary  in  their  reliability  and  validity. 

A  recent  call  for  study  of  the  construct  through  the  investigation  of 
incident  and  accident  report  data  was  well  received  (Strauch,  1995).  Strauch 
particularly  praises  the  study  of  SA  through  the  investigation  of  accident  reports 
because  of  the  logical  progression  and  in-depth  analysis  investigators  must  use 
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in  order  to  determine  the  causal  factors  affecting  an  accident,  and  to  make  future 
recommendations  (Strauch,  1995). 

Similarly,  Chappell  (1994)  also  calls  for  the  study  of  constructs  related  to 
aviation  psychology  to  use  the  wealth  of  incident  report  data  which  is  part  of  the 
Aviation  Safety  Reporting  System  (ASRS).  Chappell  describes  the  database  as  a 
compendium  of  incident  reports  gathered  from  those  who  are  involved  with  civil 
aviation  throughout  the  United  States.  Persons  submitting  reports  describe  not 
only  the  factors  involved  with  an  incident,  but  in  most  cases  how  the  incident 
occurred  (refer  to  Appendix  1).  It  is  important  to  note  here  that  the  ASRS 
database  contains  information  on  incidents,  not  accidents.  Incident  reports  are 
gathered  as  crewmembers  or  air  traffic  controllers  voluntarily  submit  reports  in 
exchange  for  immunity  regarding  the  actions  that  took  place  within  an  incident 
itself.  While  this  form  of  data  has  its  drawbacks  (biases),  it  can  be  gathered  in 
large  numbers  and  has  substantial  ecological  validity  (Chappell,  1994). 

To  understand  this  type  of  research  and  how  it  relays  valuable 
information  to  interested  parties  and  the  scientific  community,  consider  a  recent 
study  by  Owens  and  Sivak  (1993).  They  used  accident  report  data  which  is 
collected  by  the  Fatal  Accident  Reporting  System  (PARS)  in  order  to  determine 
the  role  of  reduced  visibility  in  nighttime  road  fatalities.  The  researchers 
sampled  a  group  of  reports  between  the  years  of  1980  to  1990  and  coded  a 
number  of  variables,  including  the  day  of  the  week  when  the  accident  occurred. 
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weather  and  atmospheric  conditions  during  the  accident,  and  the  lighting 
conditions  under  which  it  occurred.  The  results  showed  that  a  significant 
portion  of  the  fatalities  occurred  during  reduced  lighting  times  of  the  day,  thus 
showing  that  degraded  visibility  does  indeed  have  an  influence  upon  accidents 
and  driving  in  general.  The  usefulness  of  this  type  of  analysis  using  archival 
data  is  readily  obvious. 

The  proposed  research  seeks  to  test  Endsley's  model  of  SA  using 

voluntary  incident  report  data.  She  states  that  the  greatest  number  of  SA  errors 

will  be  in  Level  2  of  SA,  information  integration  errors.  In  incident  reports  with 

poor  SA,  this  translates  into  occurrences  of  poor  information  integration  and 

comprehension.  Based  on  Endsley's  assertions,  the  first  hypothesis  is  as  follows: 

HI:  The  most  frequent  occurrences  of  poor  SA  across  all  incident  report  data 
will  involve  Level  2  errors  (comprehension  and  integration  of  perceived 
information). 

While  Endsley's  theory  says  that  most  SA  errors  will  occur  in  Level  2, 
comprehension,  it  seems  that  this  finding  is  mOre  appropriate  for  loss  of  an 
individual's  SA.  The  discussion  of  team  SA  presented  earlier,  highlights  the 
need  for  a  shared  understanding  for  good  SA  to  exist.  If  teams  must  share 
information  in  order  for  high  quality  team  SA  to  exist,  it  seems  likely  that  Level  1 
SA  errors  would  occur  at  a  higher  proportion  for  teams.  Thus,  the  next 
hypothesis  to  be  proposed  is  as  follows: 

H2:  A  higher  proportion  of  Level  1  SA  errors  will  occur  for  team  SA  than  for 
individual  SA. 
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As  it  is  hypothesized  that  team  SA  errors  will  include  more  Level  1  SA 
errors  than  for  individual  errors,  it  seems  appropriate  to  investigate  the  possible 
factors  in  Endsley's  model  that  might  account  for  poor  SA.  For  instance, 
research  has  shown  that  the  process  of  communication  is  of  the  utmost 
importance  in  team  decision-making.  Studies  by  Orasanu  (1990)  and  Mosier  and 
Chidester  (1991)  have  shown  that  without  communication,  the  process  of 
situation  assessment,  a  precursor  for  SA,  will  breakdown.  A  team  must  perceive 
the  information  in  order  to  begin  developing  SA,  and  when  communication  does 
not  occur  in  a  team,  team  SA  should  exist  on  an  unacceptably  poor  level.  It  is 
possible  that  when  poor  team  SA  exists,  within  those  instances  that  are 
categorized  as  Level  1  errors,  communication  will  be  the  most  frequent  factor.  A 
test  of  this  assumption  is  stated  as  follows: 

H3:  Communication  will  be  the  most  frequent  factor  affecting  Level  1  SA  errors 
when  team  SA  is  poor. 

While  communication  is  thought  to  be  paramount,  other  factors  also  have 
an  influence  on  SA.  The  environmental  factors  of  workload  and  time  pressure 
should  also  have  a  large  impact  upon  SA  in  an  aviation  setting.  While  it  is 
proposed  in  H3  that  communication  will  predominate  when  team  Level  1  SA 
errors  occur,  workload  and  time  pressure  should  be  the  most  frequent  factors 
affecting  Level  2  errors.  This  seems  likely  as  both  factors,  workload  and  time 
pressure,  induce  stress  upon  operators,  causing  them  to  narrow  their  focus  (as 
individuals  or  in  teams),  leading  the  operators  to  make  premature  decisions.  As 
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stated  earlier,  as  attentional  demands  are  increased  and  working  memory  is 

constrained  by  stress  from  workload  and  time  pressure,  one  is  limited  in  one's 

abilities  to  retrieve  and  utilize  the  information  stored  in  long-term  memory. 

Thus,  this  induced  stress  from  workload  and  time  pressure  should  limit  one's 

ability  to  comprehend  and  integrate  all  of  the  information  surrounding  him  or 

her.  The  final  hypothesis  to  be  tested  by  this  research  is  as  follows: 

H4:  Workload  and  time  pressure  will  be  the  most  frequently  occurring  factors 
affecting  Level  2  SA  errors. 


CHAPTER  II 


METHOD 

Materials 

Five  hundred-ninety  ASRS  incident  reports  were  used  as  the  primary  data 
source  for  this  research.  An  example  of  this  report  form  is  provided  as  Appendix 
A.  The  reports  used  for  this  study  were  gathered  by  requesting  the  most  recent 
set  of  600  incident  reports  that  ASRS  could  provide.  A  representative  sample  of 
590  reports  were  gathered  from  the  time  period  of  November  1995  through 
March  1996  and  sent  to  the  researcher  for  analysis. 

Design 

A  2  X  3  quasi-experimental  design  was  used.  The  two  classification 
variables  of  the  study  are  the  type  of  SA  (individual  or  team),  and  the  level  of  SA 
error  (Level  1, 2,  or  3).  The  behavioral  variables  were  the  presence  of  workload 
(high  vs.  low),  time  pressure  (high  vs.  low),  and  the  quality  of  communication 
(poor  vs.  good)  on  the  flight  deck.  Operational  definitions  of  these  classification 
and  behavioral  variables  are  shown  in  the  scoring  protocol  (Appendix  B). 
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Procedure  ('Assessment  of  the  Reports') 

Each  ASRS  incident  report  was  analyzed  using  the  scoring  protocol  shown 
in  Appendix  B.  Information  from  each  report  was  gathered  concerning  a 
determination  if  poor  SA  quality  existed,  and  if  so,  the  level  of  SA  error,  and 
whether  the  incident  involved  team  or  individual  SA.  Also  information 
concerning  the  possible  explanatory  factors  affecting  the  incident  including 
instances  of  weather  problems,  equipment  problems,  distractions,  improper 
procedures,  unfamiliarity  with  the  airframe  or  area  of  flight,  night  conditions, 
fatigue,  emotional  crew  members,  communication,  workload,  and  time  pressure 
were  gathered  from  each  report.  Finally,  information  concerning  the  phase  of 
flight  when  the  incident  occurred  was  recorded. 

The  researcher  read  each  report  and  followed  the  detailed  description  for 
scoring  the  reports  as  described  in  the  protocol.  A  database  of  information  was 
compiled  for  SA  errors  and  portions  of  it  are  presented  in  Appendix  C. 

Reliability  of  Coding  Scheme 

Test-retest  and  inter-rater  forms  of  reliability  were  pertinent  to  the 
analysis  of  the  incident  report  data.  Subjective  assessments  of  the  incident 
reports  occurred  when  determining  if  a  loss  in  SA  occurred,  who  was  involved 
with  that  loss  (individual  or  entire  crew),  the  phase  of  flight  in  which  the  incident 
occurred,  and  the  factors  which  existed  during  the  incident.  The  test-retest 
reliability  coefficients  ranged  from  of  0.816  (Workload  factor)  to  0.517  (other 
factors  involved)  with  an  overall  coefficient  of  0.718.  The  inter-rater  reliability 
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check  was  completed  by  having  three  raters  (the  researcher,  a  US  Air  Force  pilot, 
and  a  General  Aviation  pilot)  complete  two  samples  of  22  reports  each  (all  44 
from  the  total  sample  population  of  the  590  reports)  in  order  to  assess  the  levels 
of  agreement  across  the  variables  of  consideration.  The  researcher  is  a  US  Air 
Force  behavioral  scientist  with  no  flight  experience.  Another  rater  is  a  Certified 
Flight  Instructor  with  a  single  engine  rating  and  300  total  hours  of  flight  time. 

The  other  rater  is  a  US  Air  Force  KC-135  Command  Pilot  with  2500  flight  hours 
occupying  crew  positions  of  command  pilot,  copilot,  and  flight  instructor.  The 
inter-rater  reliability  coefficients  ranged  from  0.794  (Workload  factor)  to  0.404 
(level  of  SA  error)  with  an  overall  inter-rater  reliability  of  0.609.  All  reliability 
coefficients  are  shown  in  Table  I.  While  these  levels  of  vary,  these  raters  did  not 
go  through  extensive  training  in  scoring.  Instead  they  read  and  scored  the 
reports  using  the  standardized  protocol  as  presented.  Thus,  training  would  likely 
increase  these  reliability  coefficients. 

While  these  reliability  coefficients  may  be  considered  "low"  in  some 
instances,  they  do  point  out  that  a  number  of  the  decisions  raters  had  to  make 
concerning  the  variables  of  interest  were  quite  difficult.  This  information  can  be 
useful  for  other  researchers  when  either  revising  or  replicating  this  form  of 
research  as  they  can  better  understand  which  aspects  of  protocol  development 
may  need  more  attention  than  others. 

It  is  also  interesting  to  note  the  lower  reliability  coefficients  regarding  CFI 
with  the  other  raters  from  Table  I.  As  this  rater  works  a  great  distance  from  the 
primary  researcher,  time  could  not  be  spent  discussing  a  large  number  of  the 
ratings  given  for  variables  within  each  of  the  sets  of  reports  used  for  reliability 
purposes.  It  is  likely  that  if  each  of  the  researchers  would  have  had  more  access 
to  each  other  to  discuss  the  ratings  given  during  each  reliability  set,  susequent 
ratings  would  have  shown  higher  agreement  levels. 
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Table  I.  Test-retest  and  inter-rater  reliability  coefficients. 


_ Reliability _ 

Variables  RE-AFP  RE-CFI  CFI-AFP  Inter-rater  Test-Retest 


SA  Error 

.896 

.542 

.458 

.632 

.896 

Level  of  SA 

.644 

.339 

.230 

.404 

.605 

Who  involved 

Incident 

.864 

.659 

.523 

.682 

.864 

Communication 

.725 

.589 

.639 

.651 

.817 

Other  Factors 

.517 

.332 

.389 

.412 

.517 

Workload 

.897 

.795 

.690 

.794 

.897 

Time  Pressure 

.897 

.727 

.690 

.771 

.795 

Phase  of  Flight 

.661 

.444 

.467 

.524 

.596 

Overall  Average  reliability  .609  .748 


RE=Researcher 
AFP=Air  Force  Pilot  Rater 
CFI=Certified  Flight  Instructor  Rater 


CHAPTER  III 


RESULTS 

Descriptive  statistics  were  gathered  from  the  590  incident  reports.  Tables 
were  developed  to  show  both  the  relative  frequency  of  the  different  levels  of  SA 
involved  in  the  incidents,  and  also  the  major  trends  within  the  data  concerning 
the  factors  involved  when  SA  errors  did  and  did  not  occur.  The  data  concerning 
the  level  SA  errors  was  coded  using  a  l-to-5  scale  which  allowed  coders  to  rate 
the  extent  to  which  any  level  of  SA  error  existed.  Any  error  rated  5  was 
considered  to  be  an  error  involving  that  level  of  SA.  Table  II  shows  the 
distribution  of  ratings. 

Based  upon  the  transformations  of  the  data  from  the  rating  scales,  it  was 
determined  that  239  incidents  did  not  involve  any  SA  error  (no  SA  rating  of  5) , 
205  incidents  involved  some  level  of  SA  error  (at  least  one  SA  rating  of  5),  and 
146  incidents  involved  bad  information  or  no  information  being  passed  to  the 
pilots,  crews,  or  ATC  controllers  involved.  For  these  last  incidents,  no  SA  error 
rating  was  assigned. 

Table  III  lists  the  factors  which  affected  incidents  when  no  SA  error 
occurred,  when  an  SA  error  was  identified,  and  when  poor  information  was 
given  to  operators  or  information  was  not  available.  The  instances  of  poor 
information  being  given  to  a  crew  member  or  when  information  was  not 
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available  were  not  considered  instances  of  SA  errors.  Endsley's  theory  (1995a) 
would  categorize  these  instances  as  Level  1  SA  errors,  but  within  the  scope  of 
this  research,  they  were  not  considered  SA  errors  because  pilots,  crews,  or  ATC 
operators  would  not  have  had  the  opportunity  to  develop  a  good  perception  and 
understanding  of  the  current  situation.  The  distinction  was  made  between 
controllable  SA  errors  and  instances  where  poor  or  no  information  was  provided 
to  the  pilot,  crew,  or  ATC  operator.  Table  III  compares  factors  affecting  incidents 
that  did  not  involve  SA  with  those  involving  poor  or  no  information  being 
available,  and  incidents  when  an  SA 

Table  II.  Frequency  of  incidents  being  rated  for  SA  errors  using  the  study 
protocol.  Each  incident  was  rated  on  every  level  of  SA. 


Scale  used:  error  absolutely  does  not  1  2  3  4  5  error  absolutely  falls 

fall  into  this  level  category  into  this  level  category 


Rating  Received 

1  2 _ 3 _ 4 _ 5 


Level  of  SA 


1 

455 

17 

18 

5 

95 

2 

468 

10 

20 

7 

85 

3 

550 

5 

8 

1 

26 

30 

error  occurred.  Tables  IV  compares  these  groups  in  terms  of  phase  of  flight 
when  the  incidents  occurred. 

Based  upon  an  examination  of  these  two  tables,  it  can  be  seen  that 
extracting  the  incidents  which  involved  poor  information  were  similar  to  those 
of  no  SA  error.  The  factors  affecting  these  incidents  as  well  as  the  incidents 
involving  no  SA  error  are  very  similar  (except  for  poor  communication,  which  is 
what  defined  the  second  group).  As  shown  in  Table  IV,  the  SA  group  is  different 
from  the  others  in  terms  of  the  phase  of  flight  of  the  incident.  Therefore,  the 
incidents  involving  no  SA  loss  and  poor  or  no  information  were  combined  (as  a 
conservative  test  of  SA)  for  a  total  of  385  incidents  for  the  remainder  of  the 
analyses  performed. 
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Table  III.  Percentage  of  reports  with  contributing  factors  identified. 


No  SA  Error 

Bad  Information 

Given 

SA  Error 

N 

239 

146 

205 

Factors 

Poor  Communication 

5.0 

36.3 

29.7 

High  Workload 

3.8 

5.4 

16.6 

High  Time  Pressure 

16.0 

22.6 

23.4 

Weather 

26.1 

21.2 

32.2 

Improper  Procedure 

17.3 

19.2 

44.4 

UnfamUiarity 

1.7 

2.7 

21.0 

Night  Conditions 

4.2 

7.5 

20.0 

Distractions 

5.5 

8.9 

20.0 

Equipment  Problems 

31.9 

43.2 

10.2 

Fatigue 

0.4 

0.0 

7.3 

Emotions 

1.3 

0.0 

1.4 
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Table  IV.  Percent  of  incidents  by  phase  of  flight. 


No  SA  Error 

Bad  Information 
Given 

SA  Error 

N 

238 

146 

205 

Phase  of  Flight 

Preflight 

7.6 

9.6 

8.3 

Taxi-out 

6.7 

3.4 

21.0 

Takeoff 

7.6 

8.90 

3.4 

Climb 

6.7 

9.6 

7.8 

Cruise 

18.5 

24.0 

8.5 

Descent 

5.5 

6.9 

8.3 

Approach 

17.6 

17.1 

16.1 

Landing 

7.1 

2.7 

4.4 

Taxi-in 

4.6 

1.4 

8.8 

Unable  to  Determine 

16.8 

15.8 

3.4 

The  following  sections  examine  the  four  hypotheses  of  this  study  and  the 
additional  analyses  which  were  performed.  The  additional  analyses  address  the 
contributing  factors  that  affected  incidents  as  well  as  the  phase  of  flight  in  which 
incidents  and  SA  errors  occurred.  Some  of  the  tests  for  both  the  hypotheses  and 
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addtional  analyses  of  contributing  factors  and  the  phase  of  flight  were  base  on 
two  series  of  Multivariate  Analyses  of  Variance  (MANOVA). 

One  series  of  MANOVAs  examined  the  pattern  of  factors  affecting  team 
SA  errors,  individual  SA  errors,  and  incidents  when  no  SA  errors  occurred.  A 
MANOVA  was  conducted  for  each  level  of  SA.  The  independent  grouping 
variable  was  based  on  team  SA  errors,  individual  SA  errors,  and  incidents 
involving  no  SA  error.  The  11  dependent  variables  were  the  factors  shown  in 
Table  III.  A  Bonferroni  significance  level  correction  was  performed  for  the  three 
MANOVAs.  The  first  MANOVA  testing  for  differences  between  the  groups  for 
Level  1  SA  was  significant,  F  (22, 908)  =  6.982,  p<0.016,  ^  =  0.247.  (Throughout 
this  study,  findings  are  considered  significant  when  p<0.05  unless  more 
conservative  post  hoc  significance  level  adjustments  were  necessary).  The 
second  MANOVA  testing  for  differences  between  the  grouping  variables  for 
Level  2  SA  was  also  found  to  be  significant,  F  (22, 908)=  9.995,  =  0.311,  as  was 

the  third  MANOVA  for  Level  3,  F  (22,  790)  =  6.355,  =  .222. 

The  second  series  of  MANOVAs  was  based  on  differences  in  phases  of 
flight  among  the  three  groups.  The  10  dependent  variables  were  the  different 
phases  of  flight  shown  in  Table  IV.  A  MANOVA  test  was  conducted  for  each 
level  of  SA  and,  again,  Bonferroni  significance  level  corrections  were  performed. 
Two  of  the  three  MANOVAs  were  significant;  for  Level  1  SA,  F(18, 932)  =  3.280, 
^  =  0.106;  and  for  Level  2  SA,  F  (18, 912)  =  4.07,  R?  =  0.125.  The  MANOVA  for 
Level  3  SA  errors  was  not  significant,  F  (18, 794)  =  1.57,  ^  =  0.054. 
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When  significant  differences  were  found  in  the  MANOVAs,  individual 
Analysis  of  Variance  (ANOVA)  tests  were  performed  on  the  factors  or  phases  of 
flight. 


Tests  of  Hypotheses 

The  first  hypothesis  predicted  that  the  most  frequent  occurrences  of 
poor  SA  would  involve  Level  2  SA  errors.  Table  V  shows  the  number  of  errors 
which  occurred  by  Level  of  SA.  A  Chi-square  test  for  goodness  of  fit  test  showed 
that  a  significant  difference  exists  between  the  frequency  of  these  types  of  errors , 
X  2  (2)  =  42.08.  The  significant  difference  results  from  the  fewer  number  of  Level 
3  SA  errors.  Level  2  SA  errors  did  not  occur  most  often.  Thus,  the  first 
hypothesis  was  not  supported. 

The  second  hypothesis  predicted  that  when  team  SA  could  be 
differentiated  from  individual  SA,  a  higher  proportion  of  Level  1  errors  would 
exist  for  teams.  The  first  row  of  Table  VI  lists  the  frequency  of  SA  errors  by  level 
for  teams  and  individuals.  A  Chi-square  for  independence  test  was  performed  to 
determine  if  these  frequencies  were  significantly  different.  It  was  found  that 
they  were  not  significantly  different,  x  ^  (2)  =  1.18.  The  second  hypothesis  was 
not  supported,  as  47%  of  the  individual  errors  involved  Level  1  SA  and  45%  of 
the  team  errors  involved  Level  1  SA. 
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Table  V.  Frequency  of  SA  errors  by  level. 


Levels  of  SA  Error _ Frequency _ Percentage  of  Errors 


1 

95 

46.1% 

2 

85 

41.3% 

3 

26 

12.6% 

1  and  2 

1 

0.5% 

1  and  3 

0 

0.0% 

2  and  3 

0 

0.0% 

1, 2  and  3 

0 

0.0% 

Level  1  SA  Error:  Failure  to  perceive  available  information 

Level  2  SA  Error:  Failure  to  comprehend  or  integrate  perceived  information 

Level  3  SA  Error:  Failure  to  predict  the  near  future  status  of  current  dynamics 
based  upon  information  comprehended 
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Table  VI.  Percent  frequency  of  individual  and  team  errors  by  SA  level. 


Individual  Team 


No  SA  Error 

LI 

L2 

L3 

LI 

L2 

L3 

N  385 

48 

39 

15 

47 

46 

11 

Factors 

Poor  Communication 

16.9 

16.7 

30.8 

20.0 

23.4 

47.8 

45.5 

High  Workload 

4.4 

10.4 

17.9 

6.7 

14.9 

23.9 

27.3 

High  Time  Pressure 

18.3 

22.9 

28.2 

40.0 

14.9 

17.4 

45.5 

Weather 

24.1 

29.2 

35.9 

33.3 

38.3 

26.1 

27.3 

Improper  Procedure 

18.1 

42.0 

53.8 

46.6 

34.0 

34.8 

100.0 

Distractions 

6.8 

22.9 

20.5 

13.3 

29.8 

13.0 

9.1 

Night  Conditions 

5.2 

27.1 

17.9 

20.0 

19.1 

17.4 

9.1 

Unfamiliarity 

2.1 

12.5 

35.9 

26.6 

10.6 

26.1 

18.2 

Equipment  Problems 

36.1 

10.4 

17.9 

6.7 

10.6 

2.2 

18.2 

Fatigue 

0.3 

10.4 

5.1 

0.0 

10.6 

4.3 

9.1 

Emotions 

0.3 

2.1 

2.6 

6.7 

0.0 

0.0 

0.0 

The  third  hypothesis  predicted  that  the  most  frequent  factor  affecting 
Level  1  SA  errors  for  teams  would  be  poor  communication.  In  order  to  answer 
this  question,  an  ANOVA  was  performed  for  Level  1  SA  errors  which  showed 
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that  poor  communication  was  not  significantly  different,  F  (2, 474)  =  0.65,  ^  = 
0.002.  Table  VI  shows  the  percent  frequency  of  poor  communication  for  team 
Level  1 SA  errors,  individual  Level  1  SA  errors  and  incidents  involving  no  SA 
errors.  While  poor  communication  affects  23.4%  of  the  team  Level  1  SA  errors  it 
is  not  significantly  different  from  the  individual  Level  1  SA  errors  or  the  no  SA 
errors  groups.  The  hypothesis  that  communication  difficulties  would  be  the 
most  frequent  factor  affecting  team  Level  1  SA  errors  is  not  supported. 

The  final  hypothesis  stated  that  the  most  frequent  factors  affecting  Level  2 
SA  errors  for  teams  would  be  workload  and  time  pressure.  Analysis  of  the 
individual  ANOVA  tests  showed  that  high  workload  was  significantly  different 
among  groups  F  (2, 464)  =  15.44,  =  0.062.  Tukey  HSD  calculations  showed  that 

it  significantly  affected  a  larger  proportion  of  team  Level  2  SA  errors  than 
incidents  which  did  not  involve  SA  errors.  However,  team  and  individual  Level 
2  SA  errors  were  not  different. 

High  time  pressure  was  not  found  to  be  a  significant  factor  affecting  team 
Level  2  SA  errors,  individual  Level  2  SA  errors,  and  incidents  involving  no  SA 
errors,  F  (2, 464)  =  1.16,  ^  =  0.005.  In  Table  V,  one  can  see  that  while  workload 
and  time  pressure  occur  in  41.3%  of  the  Level  2  team  SA,  they  are  not  the  most 
frequently  occurring  factors.  The  most  frequently  occurring  factor  for  team 
Level  2  SA  errors  was  poor  communication  (47.8%).  Thus,  the  final  hypothesis 
was  only  partially  supported. 


38 


Additional  Analyses 

As  discussed  previously,  poor  communication  accounted  for  a 
significantly  larger  proportion  of  errors  involving  team  Level  2  SA.  Additional 
tests  were  performed  to  better  understand  what  could  have  led  to  this  result. 
Instances  of  poor  communication  had  been  coded  as  involving  just  the  crew 
members  of  an  aircraft  or  involving  crew  members  and  outside  sources  such  as 
air  traffic  control  (ATC).  An  analysis  of  the  data  showed  that  only  3  (of  the  22 
total)  instances  of  poor  communication  within  team  Level  2  SA  errors  involved 
only  crew  members.  The  other  19  instances  involved  poor  communication 
between  crew  members  and  other  sources  of  communication  and  information 
such  as  ATC.  A  Chi-square  test  for  goodness  of  fit  shows  that  this  difference  in 
instances  of  poor  communication  was  significant,  x  ^  (1)  =  10.36. 

In  order  to  understand  the  if  other  possible  factors  beside  communication, 
high  workload,  and  high  time  pressure  differentially  affected  team  Level  1  SA 
errors,  individual  Level  1  SA  errors  and  incidents  involving  no  SA  errors, 
individual  ANOVA  tests  were  performed.  The  factors  of  improper  procedures 
[F  (2, 474)  =  9.42,  =  0.038],  distractions  [F  (2, 474)  =  16.76,  ^  =  0.  066  ],  night 

time  conditions  [F  (2, 474)  =  17.21,  =  0.068],  level  of  familiarity  [F  (2, 474)  = 

9.35,  R^=  0.038]  equipment  problems  [F  (2, 474)  =  12.17,  R^=  0.049],  and  fatigue  [F 
(2, 474)  =  19.11,  ^  =  0.075]  were  significant  for  Level  1  SA  errors.  Tukey  HSD 
calculations  were  performed  to  identify  differences  among  the  grouping 
variables  when  significant  differences  were  found.  The  pattern  that  emerges  is 
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that  for  both  team  and  individual  Level  1  SA  errors,  each  of  these  contributing 
factors  except  for  equipment  problems  occur  in  significantly  higher  proportions 
than  in  incidents  involving  no  SA  error.  Equipment  problems  occurred  in 
significantly  higher  proportions  of  incidents  involving  no  SA  error. 

The  same  set  of  ANOVA  tests  were  performed  for  Level  2  SA  errors. 
Improper  procedures  [F  (2, 464)  =  15.97,  =  0.064],  poor  communication  [F  (2, 

464)  =  13.88,  =  0.056,  distractions  [F  (2, 464)  =4.93,  R^  =  0.021],  night  time 

conditions  [F  (2, 464)  =  7.94,  ^  =  0.033],  familiarity  [F  (2, 464)  =  52.84,  ^  =  0.185], 
equipment  problems  [F  (2, 464)  =  13.42,  ^  =  0.055],  and  fatigue  [F  (2, 464)  =  6.69, 
Rf  =  0.028]  were  all  found  to  produce  significant  group  differences.  Tukey  HSD 
calculations  were  performed  to  identify  differences  among  the  grouping 
variables  when  significant  differences  were  found.  Poor  communication  was 
found  significantly  more  often  in  team  errors  than  individual  or  no  Level  2  SA 
errors.  The  factors  of  improper  procedures  and  distractions  occurred  in  ordy  a 
significantly  higher  proportion  of  individual  Level  2  SA  errors  compared  to  the 
other  two  groups.  The  factors  of  night  conditions,  familiarity,  and  fatigue 
occurred  in  significantly  higher  proportions  of  both  team  and  individual  Level  2 
SA  errors  than  in  the  no  SA  error  group. 

Finally,  individual  ANOVA  tests  were  performed  to  find  the  factors 
which  differentially  affected  Level  3  SA  errors.  The  results  show  that  the  factors 
of  improper  procedure  [F  (2, 405)  =  27.48,  R^  =  0.119],  poor  communication  [F  (2, 
405)  =  3.08,  Rf  =  0.15],  familiarity  [F  (2, 405)  =  18.25,  R^  =  0.083],  high  workload  [F 
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(2, 405)  =  5.86,  =  0.028],  high  time  pressure  |F  (2, 405)  =  4.53,  =  0.022], 

equipment  problems  [F  (2, 405)  =  3.47,  R^  =  0.017],  and  fatigue  [F  (2, 405)  =  8.89, 

^  =  0.042]  were  all  found  to  be  significantly  different  depending  on  whether 
they  occurred  in  a  team  SA  error,  an  individual  SA  error  or  an  incident  involving 
no  SA  error.  Tukey  HSD  tests  showed  that  the  factors  of  improper  procedures, 
poor  communication,  high  workload,  and  fatigue  all  occurred  in  significantly 
higher  proportions  for  team  Level  3  SA  errors  compared  to  the  other  two  groups. 
The  factors  of  familiarity  and  high  time  pressure  occurred  in  significantly  higher 
proportions  of  both  team  and  individual  Level  3  SA  errors. 

In  summary,  only  one  factor,  level  of  familiarity,  was  consistently  found 
more  in  SA  error  groups  than  the  no  SA  error  group  across  all  three  levels  of  SA 
error  for  both  teams  and  individuals.  Two  factors  (night  conditions  and  fatigue) 
occur  consistently  more  often  in  both  team  and  individual  Level  1  and  2  SA 
errors.  The  other  factors  of  improper  procedures,  distractions,  high  workload, 
poor  communication,  and  high  time  pressure  seem  to  affect  only  certain  levels  of 
SA  for  teams  or  individuals. 

In  order  to  understand  the  phase  of  flight  when  the  different  types  of  SA 
errors  occurred,  individual  ANOVA  tests  were  performed.  Table  VII  shows  the 
percent  frequency  of  errors  and  incidents  during  each  phase  of  flight  for  team, 
individuals  and  incidents  involving  no  SA  errors.  The  series  of  ANOVA  tests  for 
phase  of  flight  for  Level  1  SA  error  showed  differences  only  for  taxi-out  [F  (2, 

474)  =  13.56,  R^  =  0.054]  and  taxi-in  [F  (2, 474)  =  7.06,  R^  =  0.029].  The  Tukey  HSD 
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tests  revealed  that  taxi-in  was  cited  more  frequently  for  individual  SA  errors 
than  the  other  two  groups  and  taxi-out  was  cited  more  frequently  for  team  SA 
errors  than  the  other  two  groups. 


Table  VII.  Percent  of  incidents  by  SA  level,  origin  of  error,  and  phase  of  flight. 


Individual 

Team 

No  SA  Error 

LI 

L2 

L3 

LI 

L2 

L3 

N 

385 

48 

39 

15 

47 

46 

11 

Phase  of  Flight 

Preflight 

8,3 

10.4 

5.1 

20.0 

12.5 

2.1 

0.0 

Taxi-out 

6.2 

16.7 

15.4 

6.7 

25.5 

34.8 

0.0 

Takeoff 

8.1 

2.1 

2.6 

0.0 

6.4 

4.3 

0.0 

Climb 

7.8 

6.3 

12.8 

20.0 

4.3 

2.2 

18.2 

Cruise 

20.3 

18.8 

30.8 

13.3 

14.9 

17.4 

9.1 

Descent 

6.0 

8.3 

5.1 

0.0 

10.6 

13.0 

0.0 

Approach 

17.4 

8.3 

15.4 

20.0 

12.8 

17.4 

54.5 

Landing 

5.2 

4.2 

7.7 

13.3 

2.1 

0.0 

9.1 

Taxi-in 

3.4 

14.6 

2.6 

6.7 

10.6 

6.5 

9.1 

Unable  to  Determine  17.4 

8.3 

2.6 

0.0 

0.0 

2.2 

0.0 

The  set  of  ANOVA  tests  for  Level  2  SA  errors  for  phase  of  flight  showed 
that  significant  differences  during  the  taxi-out  phase  of  flight  [F  (2, 464)  =  24.17, 
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=  0.094].  Tukey  HSD  calculations  found  that  a  significantly  higher  proportion 
of  team  Level  2  SA  errors  than  the  two  other  groups  occurred  during  taxi-out 
than  in  the  other  groups. 


CHAPTER  IV 


DISCUSSION 

Based  on  the  results  of  this  study,  there  are  several  findings  meriting 
further  discussion  from  a  theoretical  and  practical  standpioint.  The  support  for 
the  hypotheses  will  be  presented  and  discussed  as  well  as  the  implications  for 
Endsley's  (1995a)  theory.  Based  on  the  analysis  of  factors  related  to  team  and 
individual  SA  new  models  will  be  presented  for  both  team  and  individual  SA. 
Finally,  conclusions  and  recommendations  will  be  presented  regarding  research 
using  incident  reports  and  future  research  should  concerning  SA. 

Evaluation  of  Hypotheses 

The  first  hypothesis  that  Level  2  SA  errors  would  be  the  most  frequent 
types  of  SA  errors  made  by  operators  was  not  supported  by  this  research. 
Instead,  it  was  found  that  Level  1  SA  errors,  perception  failures  (failure  to 
perceive  available  information  important  to  task,  person  recognizes  error  as  it 
occurs),  occurred  slightly  more  frequently.  While  there  was  not  a  significant 
difference  between  the  number  of  Level  1  and  Level  2  SA  errors,  these  findings 
present  a  somewhat  different  picture  than  what  the  literature  surrounding  SA 
depicts  (Endsley,  1995a).  While  errors  involving  the  comprehension  of  available 
information  do  happen  frequently,  as  these  results  suggest,  it  seems  that  the 
outside  influence  of  environmental  factors  such  as  high  workload,  distractions, 
night  conditions,  fatigue,  weather,  and  unfamiliarity  with  the  airframe  or  area 
have  a  large  impact  upon  the  ability  of  aviators  to  correctly  perceive  information. 
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While  Endsley  (1995a)  acknowledges  that  attention  is  limited  and  a 
myriad  of  outside  factors  which  can  affect  one's  ability  to  correctly  perceive 
available  information  abound,  this  may  be  more  of  a  problem  than  first  expected. 
Within  her  discussion  of  the  factors  which  affect  Level  1  SA  errors,  she  mentions 
that  "some  people  appear  to  be  better  than  others  at  dividing  their  attention 
across  different  tasks"  (Endsley,  1995a,  p.  55).  This  statement  has  specific 
relevance  in  light  of  results  found  as  numerous  factors  are  working  against 
aviators  as  they  seek  to  divide  their  attention  across  tasks.  From  a  practical  view, 
these  results  present  a  need  for  developing  more  specific  training  strategies  in 
helping  aviators  manage  SA  at  its  basic  level,  dividing  one's  attention  in  order 
perceive  needed  information.  From  a  theoretical  standpoint,  the  finding  that 
Level  2  SA  errors  are  not  the  most  frequent  SA  errors  means  that  more  focus 
should  be  given  to  the  loss  of  perception  when  modeling  SA  errors. 

Proctor  and  Van  Zandt  (1994)  describe  a  number  of  strategies  which  are 
advocated  when  trying  to  help  train  operators  in  increasing  there  abilities  for 
divided  attention.  First,  these  researchers  state  that  operators  must  be  given 
ample  practice  in  dividing  their  attention  across  modalities  during  simulation 
tasks.  This  practice  in  dividing  one's  attention  across  modalities  should  help  to 
operators  become  more  proficient  at  obtaining  as  well  as  understanding 
information  in  dynamic  environments.  Further,  these  researchers  state  that 
operators  should  be  given  training  in  priortizing  and  planning  for  divided 
attention.  Here,  the  researchers  advocate  helping  operators  develop  an  action 
plan  for  dealing  with  information  when  that  information  comes  from  various 
sources.  Finally,  Proctor  and  Van  Zandt  support  delegating  duties  between 
operators  so  that  a  smaller,  more  defined  amount  of  information  is  attended  to 
by  each  operator. 
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The  second  hypothesis  stated  that  for  teams,  a  higher  number  of  Level  1 
SA  errors  would  exist.  This  hypothesis  was  not  supported  by  this  research. 
Instead,  it  was  found  that  for  teams,  the  number  of  errors  were  fairly  evenly 
distributed  between  Level  1  and  2  SA  errors,  while  Level  3  SA  errors  occurred 
least  often.  This  results  parallels  the  findings  from  the  previous  hypothesis.  This 
pattern  seems  to  show  that  for  team  SA,  both  perception  of  environmental 
information  and  comprehension  of  that  information  are  equally  important 
problems.  It  was  thought  that  team  SA  would  only  exist  if  team  members  shared 
information  between  themselves,  thus  leading  to  a  higher  proportion  of 
perception  problems.  What  this  result  indicates  is  that  teams  have  equal 
problems  in  both  perceiving  or  sharing  information  and  in  building  a  team 
shared  mental  model. 

The  recent  research  by  Salas  et  al.,  (1995)  elaborates  on  the  importance  of 
the  team  shared  mental  model  for  good  team  SA.  Clearly  the  viewpoint 
provided  by  these  researchers  hinges  on  the  notion  of  sharing  the  information 
perceived  and  working  to  understand  that  information  as  a  team.  Salas  et  al., 
(1995)  report  that  training  for  team  SA  should  encompass  both  critical 
information-seeking  behavior  for  teams  and  training  on  team  competencies  for 
crews  which  rotate  members  regularly.  Both  of  these  recommendations  are 
supported  by  this  research  because  teams  have  equal  difficulty  both  in  gathering 
and  also  comprehending  the  information  needed  for  good  decision-making. 

The  third  hypothesis  stated  that  poor  communication  would  be  the  most 
frequent  factor  affecting  team  Level  1  SA.  This  hypothesis  followed  from  the 
second  hypothesis.  It  was  thought  that  team  Level  1  SA  would  require  good 
communication  before  good  team  perception  of  the  current  situation  would  exist. 
While  the  data  from  this  study  do  not  support  this  hypothesis  entirely,  it  does 
support  the  importance  of  good  communication  for  team  perceptions.  Poor 
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communication  was  found  to  be  a  contributing  factor  in  23.4%  of  team  Level  1 
SA  errors.  Thus,  it  occurs  in  a  fairly  large  number  of  incidents  involving  team 
Level  1  SA  errors. 

While  poor  communication  was  not  found  to  be  the  most  frequent  factor 
affecting  team  Level  1  SA,  it  was  found  to  be  the  most  frequent  factor  affecting 
team  Level  2  SA,  comprehension  of  information.  Aviation  crews  are  forced  to 
understand  a  great  deal  of  information  pertaining  to  changing  weather 
conditions,  the  status  of  aircraft  around  them,  as  well  as  any  and  all  information 
pertaining  to  their  current  heading,  speed,  and  altitude.  It  seems  plausible  that 
good  communication  would  be  necessary  for  sufficient  understanding  of  a 
developing  situation.  Those  crews  which  have  spent  many  hours  flying  together 
on  numerous  occasions  may  develop  a  shared  mental  model  based  on  reading 
instruments  alone  without  requiring  overt  verbal  communication.  This  finding 
that  poor  communication  is  such  a  large  factor  affecting  team  Level  2  SA 
supports  the  current  research  which  is  being  conducted  on  improving  crew 
communication  for  developing  that  shared  problem  model  of  a  developing 
situation  (Salas  et  al.,  1995).  By  focusing  efforts  on  understanding  team 
communication  processes,  we  can  develop  stronger  training  programs  to  support 
team  comprehension  of  problem  situations. 

The  final  hypothesis  of  this  study  stated  that  high  workload  and  time 
pressure  would  be  the  most  frequent  factors  affecting  team  Level  2  SA.  The 
results  showed  partial  support  for  this  hypothesis.  High  workload  was  found  to 
be  a  significant  factor  affecting  team  Level  2  SA  (occurring  in  23.9%  of  team  Level 
2  SA  errors).  Since  high  workload  was  found  be  to  a  significant  factor  affecting 
team  Level  2  SA,  support  is  given  to  the  model  presented  by  Endsley  (1996) 
where  she  describes  how  poor  SA  and  high  workload  often  occur 
simultaneously.  She  describes  that  as  the  information  and  demand  of  tasks 
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becomes  too  great,  the  operator  begins  to  lose  the  ability  to  integrate  all  of  the 
available  information  (Endsley,  1996).  This  view  is  supported  by  the  present 
research. 

While  high  workload  was  found  to  be  a  significant  factor  affecting  team 
Level  2  SA,  the  same  finding  was  not  true  for  high  time  pressure.  While  high 
time  pressure  was  found  to  be  a  contributing  factor  in  17.4%  of  the  team  Level  2 
SA  errors,  this  trend  is  not  different  from  the  amount  of  incidents  without  SA 
errors.  High  time  pressure  seemed  to  exist  across  many  of  the  590  incident 
reports.  It  seems  that  the  general  level  of  high  time  pressure  for  most  aviation 
incidents  masks  its  significance  for  team  SA.  This  is  a  potentially  interesting 
finding  because  previous  research  into  the  factors  affecting  aviation  accidents 
does  not  mention  the  factor  of  time  pressure  (NTSB,  1994).  If  high  time  pressure 
remains  fairly  constant  across  SA  errors  and  more  general  incidents  alike,  it 
seems  to  be  an  organizational  issue  which  must  be  addressed.  Aviation 
administrative  bodies  should  investigate  methods  for  reducing  time  pressure 
within  the  aviation  environment. 

Evaluation  of  Additional  Analyses 

While  the  thrust  of  this  research  centered  around  the  main  hypotheses 
discussed  above,  additional  analyses  were  performed  in  order  to  understand 
more  about  the  nature  of  SA  errors.  Much  of  these  findings  will  be  discussed  in  a 
later  section  of  the  discussion  where  new  models  of  SA  errors  are  presented. 
What  follows  is  a  discussion  of  the  phase  of  flight  when  SA  errors  occur,  and  a 
discussion  of  other  findings  concerning  poor  communication. 

It  was  found  that  a  significantly  higher  proportion  of  team  Level  1  and  2 
SA  errors  occurred  during  taxi-out.  During  this  particular  phase  of  flight,  crews 
are  performing  numerous  last  minute  checklist  procedures,  communicating  with 


48 


ATC  and  ground  control,  as  well  as  maneuvering  the  aircraft  for  positioning  for 
takeoff.  This  is  a  very  busy  phase  of  flight  and  the  increased  workload  could  be 
seen  as  a  possible  causal  factor.  However,  it  also  found  that  poor  communication 
occurred  in  a  significantly  higher  proportion  of  team  Level  2  SA  errors  and  most 
of  those  poor  communication  interactions  took  place  between  crews  and  ATC.  It 
seems  very  possible  that  the  poor  communication  that  was  occurring  between 
crews  and  ATC  was  a  main  determinant  of  the  subsequent  SA  loss  among  crews. 

Looking  back  to  the  individual  ASRS  reports  in  question  when  poor 
communication  between  crews  and  ATC  occurred,  it  was  found  that  many 
instances  of  missing  "hold  short  lines"  occurred.  These  are  stopping  areas  placed 
just  outside  of  active  taxiways  so  that  aircraft  do  not  conflict  with  each  other. 

This  could  mean  that  ATC  personnel  and  crews  are  not  sharing  the  same  mental 
model  or  understanding  of  where  those  hold  short  lines  are  and  where  the  crews 
should  be  holding.  This  particular  example  shows  how  critical  explicit 
communication  is  for  the  accomplishment  of  airline  operations.  Possible 
solutions  to  this  problem  do  exist.  Since  there  is  some  discrepancy  in  the 
terminology  used  to  tell  crews  where  they  should  be  holding,  a  simple  solution 
would  be  for  this  terminology  to  be  standardized  industry  wide. 

Implications  for  Endsley's  Theory 

This  research  relied  on  Endsley's  (1995a)  theory  of  SA.  First,  this  theory 
has  been  shown  to  be  extremely  useful  in  shaping  a  measurement  instrument  for 
gather  information  about  SA  loss  from  incident  report  data.  The  definitions  and 
model  presented  by  Endsley's  (1995a)  theory  have  been  clear  enough  to  use 
when  scoring  a  variety  of  narrative  reports.  The  theory  also  allows  for  an 
interpretation  of  the  subsequent  data. 
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Second,  Endsley's  model,  this  researcher  developed  very  specific 
definitions  of  Level  1, 2,  and  3  SA  errors  as  well  as  relating  how  numerous 
factors  affect  those  errors  for  both  teams  and  individuals.  Endsley  (1995a)  does  a 
very  good  job  of  describing  SA  errors  across  the  specific  levels  of  SA.  These 
definitions  and  examples  served  as  the  main  guidelines  for  the  incident  scoring 
protocol  used  in  this  study.  Thus,  the  theory  presents  very  useful  concrete 
examples  of  what  an  incident  would  contain  when  SA  is  poor. 

Third,  Endsley's  theory  is  extremely  useful  from  a  researcher's  point  of 
view,  it  does  have  some  shortcomings  when  being  used  to  evaluate  SA  error  or 
loss.  This  theory  does  not  describe  at  great  lengths  the  subtle  differences  which 
exist  between  team  and  individual  SA  loss.  Instead,  the  theory  gives  general 
descriptions  of  the  requirements  for  team  SA  ("degree  to  which  every  team 
member  possesses  the  SA  required  for  his/her  responsibilities",  Endsley,  1995a, 
p.  39)  and  descriptions  of  the  nature  of  Level  1, 2  and  3  SA  errors.  This  theory, 
however,  does  a  somewhat  poor  job  of  describing  the  absolute  importance  of 
good  communication  for  the  development  of  team  SA  within  its  higher  levels. 

Endsley  describes  that  good  communication  may  or  may  not  be  needed 
for  good  Level  2  and  3  SA  as  shared  mental  models  and  scanning  strategies  may 
circumvent  the  need  for  explicit  communication  (Endsley,  1995a).  She  also  cites 
work  that  has  shown  that  more  efficient  communication  is  a  real  key  for  good 
team  SA  in  the  higher  levels  (Endsley,  1995a).  What  this  research  has  shown  is 
that  without  explicit  communication  between  crew  members  and  those  outside 
the  flight  deck  (ATC),  poor  team  SA  is  very  likely  to  exist.  Since  47.8%  and 
45.5%  of  the  team  Level  2  and  3  SA  errors,  respectively,  involve  poor 
communication,  the  theory  must  underscore  and  specifically  list  the  factor  of 
communication  as  a  moderating  variable  for  the  development  of  team  SA. 
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New  Models  of  Individual  and  Team  SA  Loss 

Since  Endsley's  theory  gives  only  models  of  the  components  of  SA  and 
lists  only  some  of  the  factors  which  may  have  an  impact  upon,  new  models  of 
both  individual  and  team  SA  loss  should  be  developed.  The  present  research 
allows  for  the  development  of  such  models  because  it  provides  data  concerning 
the  relative  proportions  of  external  factors  affecting  poor  SA  for  both  individuals 
and  teams.  The  development  of  these  models  does  not  detract  from  those  which 
have  already  been  published.  Instead,  it  adds  to  the  knowledge  base 
surrounding  poor  SA. 

Figure  2  presents  the  differences  in  the  factors  which  affect  incidents 
involving  no  SA  error  and  factors  affecting  incidents  involving  SA  errors.  This 
figure  shows  the  impact  of  external  factors  upon  SA  errors.  While  the  incident 
report  data  studied  does  not  allow  a  researcher  to  understand  the  effects  of  one's 
abilities,  training,  experience  upon  SA  errors,  it  does  allow  one  to  understand  the 
factors  which  affect  higher  proportions  of  SA  loss  incidents  than  incidents 
involving  no  SA  error.  One  can  see  that  many  more  factors  come  into  play  when 
SA  is  considered  to  be  poor.  While  this  research  focused  upon  the  relative 
impact  of  three  of  these  factors  upon  SA  (poor  communication,  high  workload, 
and  high  time  pressure),  it  is  very  evident  that  at  least  six  other  factors  are 
working  against  aviation  personnel  in  the  fight  to  maintain  peak  SA.  With  this 
information  and  the  finding  that  205  of  590  incidents  involved  SA  at  some  level, 
one  can  see  that  SA  must  receive  continued  attention  from  both  a  research 
(theory)  and  organizational  (practical)  standpoint. 


Incident  Involving  No  SA  Error 


Incident  Involving  an  SA  Error 


Figure  2.  Differences  in  factors  affecting  SA  error  incidents  and  incidents 
involving  no  SA  error. 
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Figure  3  presents  the  factors  which  were  found  to  have  a  large  effect  upon 
poor  individual  SA  at  all  three  levels.  It  is  important  to  note  that  common  factors 
occur  within  all  three  levels  of  individual  SA  loss.  The  factors  of  performing 
improper  procedures  and  unfamiliarity  (with  the  airframe  or  systems  being 
used,  other  crew  members,  or  a  geographical  area)  seem  to  affect  almost  all  three 
levels  of  individual  SA.  This  is  not  the  first  occurrence  of  these  factors  having  a 
large  effect  upon  aircraft  incidents  and  accidents.  The  NTSB  (1994)  found  that  in 
the  37  fatal  accidents  occurring  between  1987  and  1990, 24.2%  of  the  errors 
involved  in  these  accidents  involved  improper  procedures,  while  73%  of  the 
accidents  (in  which  information  was  available)  involved  unfamiliarity  of  the 
crew  members.  With  this  information,  it  is  imperative  that  aviation  management 
truly  focus  upon  both  the  training  procedures  used  in  aviation  and  the  practices 
with  which  it  assigns  personnel  to  flights. 


•high  time  pressure 
•unfamiliarity 


improper  procedures 
night  conditions 
unfamiliarity 
fatigue 
distractions 


Figure  3.  A  model  of  individual  SA  errors. 


While  Figure  3  presented  a  model  of  poor  individual  SA,  Figure  4  paints  a 
somewhat  different  picture  of  the  factors  which  affect  poor  team  SA.  This  model 
shows  that  the  additional  factors  of  high  workload,  poor  communication,  high 
time  pressure,  and  fatigue  seem  to  affect  poor  SA  when  a  team  is  involved.  This 
finding  is  not  particularly  surprising  in  the  light  of  past  research.  Aviation 


research  into  the  factors  associated  with  human  error  have  traditionally 
considered  these  factors  to  be  important  for  study  and  understanding. 
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•high  workload 

•night  conditions 
•unfamiliarity 
•fatigue 
•distractions 
•improper  procedures 


•improper  procedures 
•unfamiliarity 

•poor  communication 
•high  workload 
•high  time  pressure 
•fatigue 


What  is  surprising  is  that  these  factors  differentially  affect  SA  when  the 
SA  loss  involves  the  complex  interactions  of  teams  or  crews.  Because  good  team 
work  is  necessary  within  the  aviation  environment  as  well  as  most  other  work 
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environments,  it  is  important  to  focus  upon  training  strategies  that  will  work  to 
help  the  team  deal  with  these  factors.  Such  strategies  could  include  the 
development  of  shared  mental  model  training  to  help  teams  deal  with  the 
pressures  of  high-workload  situations  when  they  are  compounded  by  time 
pressure.  By  lessening  the  cognitive  burdens  on  the  operators,  teams  can  focus 
upon  their  shared  understanding  of  the  situation,  and  less  upon  trying  to 
determine  what  the  other  team  members  believe  is  occurring.  Teams  can  also 
work  on  efficient  communication  strategies  (using  clear  language,  only  stating 
necessary  and  important  information,  and  providing  the  information  in  a  timely 
fashion)  for  producing  necessary  information  among  team  members  before  it 
becomes  an  essential  commodity.  Finally,  in  order  to  address  fatigue, 
administrators  must  work  to  help  develop  more  worker-centered  schedules  for 
aviators  and  other  types  of  employees  supplemented  with  regular  rest  periods. 

Value  of  Incident  Report  Data 

The  research  conducted  here  has  outlined  the  usefulness  of  incident  report 
data.  With  all  of  its  drawbacks  considered  (reporter  biases,  lack  of  causality  from 
archival  data,  and  skewed  distributions  of  reports),  the  incidents  analyzed  have 
provided  a  wealth  of  information  regarding  poor  SA,  when  it  occurs,  the  factors 
involved,  and  how  we  can  work  to  address  these  issues.  This  type  of  data  is 
readily  available  and  can  be  solicited  in  mass  quantities.  It  is  as  an  extremely 
useful  source  of  information  when  trying  to  understand  the  factors  which 
aviators,  air  traffic  controllers,  ground  personnel,  mechanics,  dispatchers,  and 
other  personnel  feel  effect  a  loss  of  SA. 

While  this  type  of  data  is  extremely  useful,  it  does  have  some  drawbacks. 
One  of  the  most  important  drawbacks  to  consider  is  the  fact  that  this  data  can  be 
presented  to  a  reader  in  any  number  of  forms.  This  means  that  the  reporters  will 
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give  the  information  they  feel  is  important  using  their  own  personal 
communication  skills.  This  often  leads  to  a  wide  variety  of  possibilities  of 
interpretations  of  the  data  as  standardization  is  lost.  For  example,  it  was 
surprising  to  find  the  low  inter-rater  agreement  when  trying  to  determine  the 
phase  of  flight  in  which  an  incident  occurred.  Each  of  the  raters  insisted  that  this 
should  have  been  an  easy  goal  to  achieve.  However,  because  the  incident  reports 
can  be  presented  in  basically  any  form  of  narrative  style ,  it  was  difficult,  for 
example,  to  determine  if  the  reporter  was  describing  the  beginning  of  an 
approach  or  simply  a  descent  when  configuring  the  aircraft  just  prior  to  final 
approach.  Incident  report  data  can  vary  in  describing  a  situation  or  phase  of 
flight  and  it  becomes  very  difficult  to  standardize  a  scoring  protocol  to  analyze 
this  type  of  data. 

One  other  drawback  to  this  type  of  data  is  that  it  becomes  very  difficult  to 
understand  the  higher  level  cognitive  states  of  operators  based  on  their  verbal 
representation  of  a  given  incident.  This  fact  could  have  affected  the  results  of 
this  study  because  it  was  difficult  to  identify  if  a  Level  3  SA  error  had  occurred 
unless  the  reporter  could  clearly  define  the  thought  processes  in  the  incident 
report  itself.  It  is  entirely  possible  that  more  Level  3  SA  errors  are  occurring  in 
civilian  aviation  not  simply  because  they  are  not  reported,  but  because  the 
language  of  the  reports  does  not  allow  the  easy  identification  of  a  Level  3  SA. 
Because  these  reports  are  based  on  the  thoughts  of  aviation  personnel,  the 
drawbacks  of  a  narrative  format  could  be  masking  the  existence  of  certain  types 
of  errors. 


Conclusions  and  Recommendations 

There  are  two  readily  evident  conclusions  which  can  be  drawn  from  this 
research.  First,  SA  is  a  real  factor  affecting  the  everyday  activities  of  civilian 
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aviation.  This  is  seen  through  both  the  sheer  number  of  reports  involving  SA 
errors  as  reported  by  actual  aviation  personnel  and  findings  that  show  SA  errors 
involve  different  factors  from  incidents  not  involving  SA  errors.  Both  laboratory 
and  field  research  should  continue  to  better  understand  this  construct  and  how 
human  factors  engineering  and  training  can  be  used  to  support  SA. 

A  second  conclusion  is  that  the  theory  of  SA  developed  by  Endsley 
(1995a)  has  ecological  validity  and  is  very  useful  in  capturing  SA  errors.  While 
this  theory  may  need  some  refining,  it  does  describe  the  critical  components  of 
SA  or  how  errors  involving  SA  may  look.  Using  the  information  from  this  study 
and  others,  the  theory  should  continue  to  be  improved  for  practical  use  within 
not  only  aviation,  but  other  fields. 

While  the  factors  of  poor  communication,  high  workload  and  high  time 
pressure  were  the  primary  factors  of  this  research,  considerable  amounts  of 
information  have  been  gained  concerning  the  impact  of  other  factors  affecting  SA 
errors.  It  was  found  that  these  factors  have  differential  effects  upon  SA  in  its 
different  levels  for  both  teams  and  individuals.  This  information  should  be 
examined  and  used  further  because  it  is  based  on  direct  reports  of  incidents 
involved  with  SA  loss. 

There  are  also  several  recommendations  which  could  be  made  for 
improving  this  research.  The  main  way  in  which  this  research  could  be 
improved  is  through  a  more  developed  scoring  protocol  for  analyzing  the 
incident  reports  and  by  giving  raters  more  training.  These  considerations  would 
undoubtedly  increase  the  reliability  of  the  scoring  procedure.  Also,  the  reports 
used  in  this  study  came  from  a  specific  time  period  when  weather  can  be 
extremely  bad  in  the  United  States.  Thus,  future  work  should  be  conducted 
using  reports  which  are  randomly  sampled  from  numerous  time  periods 
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throughout  the  year  to  control  for,  among  other  factors,  the  possible  effects  of 
weather. 

Future  research  should  also  seek  to  replicate  the  findings  of  this  study.  As 
the  investigation  of  SA  and  SA  errors  is  a  fairly  new  research  area,  more  research 
should  be  conducted  to  confirm  these  findings  so  that  the  information  can  be  put 
to  practical  use.  However,  while  these  findings  may  be  new,  they  are  extremely 
important  because  they  have  the  potential  to  impact  SA  for  the  safe  operation  of 
flight.  While  not  one  person  died  in  any  of  the  590  incident  reports  used  in  this 
study,  incidents  can  easily  become  accidents.  This  research  has  specific 
applications  for  flight  safety. 


APPENDICES 


60 


Appendix  A 

Sample  ASRS  Incident  Report  Form 


1'UA‘Jt:  I  tLL  HI  APPIIOPIUAIU  SPACES  ANU  CHECK  ALL  11 EMS  WHICH  API'LY  lO  IHi:;  (,VI  HI  OH  SHUAHOH. 


I  II '-.I  t  )Mir.<*r 
tiilul  Kynui 
•  pilol  tiol  tlymi) 

■  t  )I1h:i  CiL’vvinoiiibc’i 


FLYING  TIME 

ciriTfRCATES/nATlNGS 

lolal _ 

.his. 

i-'  student 

pirvalt.* 

'  •  commercial 

«' All’ 

Iasi  90  days . 

his. 

‘•insliumenl 

"Cn 

"mullicrtyiite 

' « I  vi: 

lime  In  lypc . . . 

hrs. 

ri;ir,s  A  (PCA)  o  Special  Use  Airspace 

• '  CInr;;;  H  { I CA)  o  airway/routo _ 

tdMssC(AMSA)  o  unknown/ollier _ 

Class  0  (Conliol  Zone/A  IA)  _ 

Class  t  ((ipaeial  Coiiliollf;d)  _ _  _  __  _ 

Class  O  (UiiconlioHotl)  _ 


WEATHEn  LIGUl 

oVMC  oice  (.^dayliy 

olMC  osnow 
o  mixed  o  (urbulence 
o  marginal  o  Islorm  ceiling 

o  rain  owindshear  visibilily 


AinCRAFT  1 


I  vpn  el  Alicrall 
(Make  Muilol) 


FUghl  plan 


CoiUiol  status 


oEFiS 

(Your  Airciatl) 

o  FMS/l-MC 

o  air  carrier 

o  military 

o  corporate 

commuter 

o  private 

o  other 

'‘passenger 

o  training 

o  business 

cargo 

’>  pleasure 

unk/ollicr 

'» VFR 

osvrn 

o  none 

o  IFR 

o  DVFR 

o  unknown 

<*laxi 

o  cruise 

o  landing 

u  takeoll 

<)  descent 

o  missed  apeh/GAR 

o  climb 

o  approach 

o  olher 

o  visual  apeh 

o  on  vector 

oon  SID/STAR 

'■>  controlled 

o  none 

o  unknown 

«>  no  radio 

o  radar  advisories 

iM}.  . ;  _  "aTC  EXPERiENCE  H 

t’P'’vale  '*I-RL  u  Ucvelopmonlal 

•’  A I R  ladar  _  . . yrs. 

‘'Cl-I  nun  radar _ _yrs. 

"I'/li  supeivisoiy _ yrs. 

_  military _ yrs, 

_ ^LIGli  l/ViSluiLITY  ATCMDVISORY  SERV.I 

u  daylight  n  nicjlit  o  local  o  center 
tidnvvn  udusk  C)  ground  oFSS 

ceiling  leet  IT" 

. -  odup  oCTAF 

visibilily  miles  Name  ol  ATC  Facility: 

rWR  _  loot  _ 

AIRCRAFT  2  I 


(OlluM  Aiicrall) 

oaircanier  o  military 

o  commuter  »»  private 

o  passenger  '’training 

<>  caitjo  opieasurr 


o  taxi 
o  takeoll 
o  climb 


•approach  o  other 


o  EFIS 
oFMS/FMC 

o  corporate 
o  other _ 

o  business 
o  unk/olher _ 

o  none 
o  unknown 

o  landing' 

o  missed  apeh/GAR 


o  visual  apeh  o  on  vector 

oconliolled  mione 

o  no  I  adio  i  *  radar  adv 


on  vector  o  on  SID/STAR 

none  o  unknown 

radar  advisories 


11  more  than  two  alrcrall  were  Involved,  please  describe  the  additional  airciatl  In  the  “Describe  Evcnl/Siluallon”  section. 
_  LOCATION  CONFUnTR 


Altitude  . . o  MSL  oAGL 

Distance  and  radial  lioni  ail  poll,  NAVAID,  or  other  llx _ 

Ncnicst  Cily/Slale _ 


_ CONFLICTS  I 

Eslimaled  miss  distance  in  feet;  horiz  _  verl  _ 

Was  evasive  action  taker  u  Yes  o  No 

Was  TCAS  a  (actor?  o  TA  o  RA  o  No 

Did  GPWS  activate?  o  Yes  o  No 


1 _ DESCRIBE  EVENT/SiTUATION _  .  } 

K....,.(t,q  tfi  minrj  the  lopics  shown  below,  discuss  Ihoso  which  you  leel  are  rdevanl  and  anything  else  you  think  is  impotlani  Include  what  you  believe  realty  caused  Ihe 
piohinm.  and  whal  can  be  done  lo  pteveni  a  recurrence,  ot  coned  the  situalion.  (CONTINUE  ON  THE  OrUER  SIDE  AND  USE  ADDITIONAL  PAPER  IF  NEEDED) 


CHAIN  OF  EVENtS 

•  How  the  proljicin  arose  ■  How  il  was  discovered 

Corrifibuling  laclors  -  Corrective  actions 


NASA  ARC  277B  (Jarmary  199/i) 


HUMAN  PEHFOnMANCE  CONSIDERATIONS 
-  Percepliotrs.  judgmonls.  dt»cir>ions  -  Actions  or  inadiotts 
'  Factors  allecling  the  qunlily  ol  Iturnan  portortnance 

n  U.S.  CXJVEftNMENrPfUNriNO OFFICE  l994-5«5'eoe 


FIRST  FOLD 
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NA I K  )NA1.  Al.  I  lONAU  I  ICS  AND  SPACIi  AUMINIS 1 1  lAl  lUN 

flAliA  li;r,  ;iti  Avinlion  Salnly  Hopotlinq  Syslotn  (ASI^S)  lo 

'tli'tiiily  tssuw  HI  ihn  aviation  syslnni  whicli  neod  lo  be  adUiessed.  The 
|nn(irani  nl  which  ihis  system  is  a  pari  is  described  indelail  in  FAA  Advisory 
‘  . .  Your  nssislance  in  inlorniing  US  nboul  such  issues  is 

•  '‘■.f;i  -1111.11  In  tl  k;  success  ot  Iho  pi  ogratn.  Please  iill  oul  this  postage  li  ee  lortii 
as  cnuipliHely  as  |)Ossible.  (old  it  and  send  it  direclly  to  us. 

111.'  iriliiHiKiiiiin  you  (Hovide  on  Ihe  idonlily  strip  will  be  used  only  il  NASA 

•  l.'ir.miitii.';  ihal  il  is  nec.rr.sary  tuenniaci  you  (oi  (uilht'r  inlurniation.  I  HIS 
II  'I  Mill  Y  S I  HIP  Wil  t  ni:  I  It  1 1 IHNIU)  Oil  ll:C  I LY  I O  YOU.  I  he  return  ol 
|hi«  itl«‘Mtily  sliip  .issuies  your  aiionyniity. 


AVIA  I  ION  r.AI  1. !  Y  I  li:i>Ui  1 1 ING  SYS  I  tM 

SectionO  1 .25  ol  the  f  oderal  Aviation  (legulations  ( 1 4  CPH  9 1.25) prohibits 
reports  liled  with  NASA  liuni  Ih'iiuj  used  (or  FAA  onloiceiner^t  purposes. 
This  report  wilt  not  be  made  availaljle  lo  Ihe  FAA  lor  civil  penally  or 
certilicale  acliot>s  lor  violations  ol  the  I-  ndtu  at  Air  I  legulations.  Your  identity 
strip,  stanrped  by  NASA,  is  piool  that  you  have  subrniltod  a  report  to  the 
Aviation  Safety  lleportitKi  System.  We  can  only  return  the  strip  to  you, 

however,  il  you  liave provided  a  riiaitin.  I  address,  tqualty  tmporlanl.wecan 
often  obtain  addiliortal  usetui  inloi  mation  il  our  safety  analysis  can  talk  with 
you  direclly  by  lolephone  For  ihis  reason,  we  have  roqucsicd  telephone 
numbers  where  we  may  reach  you, 

lliank  you  lor  your  contiibuliuii  lo  aviation  salcly. 


AJi  >//  Aii.t'n/\n  Accm.Nif:  siiouu}  not  be  hepoihedon  mis  foiim.  suciicvcnis  snout  u  ue  filed  with  TiiENATinuAi 
IIIANSroniA  I  ION  SAFE  l  Y  aOAIW  AS  nEOmiEO  UYNTSU  noguMu,,,  830.5  (■wcniiuoE,  national 


UESeniDE  EVENt/SIlUAllON  (continued): 
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Appendix  B 

Incident  Report  Scoring  Protocol 

Fill  in  the  blanks  with  appropriate  information  or  check  which  option  is 
applicable. 

Report  Number _ 

1)  Determine  if  poor  S  A  quality  exists  in  the  report. 

-the  individual  or  team  may  fail  to  get  needed  information,  fail  to  understand 
what  perceived  information  means,  or  fail  to  make  a  prediction  of  actions  based 
on  an  understanding  of  the  information  received 


2)  Determine  who  was  involved  primarily  with  the  failure  to  maintain 
Situation  Awareness.  Use  the  following  categories  (aircrew,  gndcrew,  tower) 


3  a)  Categorize  the  Situation  Awareness  error  using  the  scale  provided  by 
rating  the  degree  to  which  the  error  fits  into  each  of  the  three  Levels  listed.  As 
reports  may  have  numerous  SA  errors,  please  rate  how  the  most  prominent  or 
overarching  S A  error  fits  into  each  of  the  categories  listed. 

Level  1  error:  perception  problem;  failure  to  perceive  available  information 
important  to  task,  person  recognizes  error  as  it  occurs 

error  absolutely  does  not  1  2  3  4  5  error  absolutely 

fall  into  this  level  falls  into  this 

category  level  category 


Level  2  error:  comprehension  problem;  inability  to  integrate  information, 
misreading  of  perceived  cues  available 

error  absolutely  does  not  1  2  3  4  5  error  absolutely 

fall  into  this  level  falls  into  this 

category  level  category 


Level  3  error:  projection  of  future  status  problem;  lack  of  mental  simulation  of 
actions  before  decision  is  made,  understands  information  presented  but  fails  to 
project  how  that  will  affect  the  flight  (seeing  mountain,  understanding  it  will  not 
be  cleared,  yet  flying  toward  it  anyway) 

error  absolutely  does  not  1  2  3  4  5  error  absolutely 

fall  into  this  level  falls  into  this 

category  level  category 
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Provide  a  brief  description  of  the  SA  error  committed: 


3  b)  Was  bad  information  passed  to  the  operators  in  the  incident,  or  was  no 
information  readily  available  for  SA  to  exist? 

Yes _  No _ 

4)  Did  only  one  person  lose  awareness  or  the  entire  crew  in  the  SA  error? 

Team  _ _  Individual _  Unable  to  Determine _ 


5)  Was  poor  communication  a  factor  affecting  the  SA  error?  Is  communication 
cited  by  the  reporters  as  a  factor  affecting  the  SA  error  or  is  it  apparent  that  it  is 
a  factor? 

Poor  communication:  Crew  members  state  that  people  are  not  sharing 
information  that  is  necessary  for  flight  task  accomplishment  (e.g.  PNF  not 
relaying  information  from  tower  to  PF)  or  not  sharing  information  in  a  timely 
manner 

Good  Communication:  Crew  members  state  that  they  are  sharing  information  in 
a  timely  fashion  (before  or  as  the  information  is  needed) 

Unable  to  determine:  no  apparent  evidence  is  provided  relating  that  poor 
communication  was  a  factor  affecting  the  error. 

Poor  communication _  Poor  communication  between  members  in  the 

cockpit _ 

Poor  commtmication  between  crew  and  others  outside  the  cockpit  (ie,  tower) _ 

Good  Communication _ 

Unable  to  Determine _ 


64 


6)  Check  off  the  other  factors  noted  by  the  crew  members  which  you  feel 
affected  the  S A  error. 


.Improper  procedure: 

Did  an  individual  or  team 
perform  actions  which 
should  not  have 
occurred  during  the  flight? 

_  Aircraft  equipment 
malfunctions: 

Does  some  form  of  equipment 
malfunction  occur  prior  to  or  in 
conjunction  with  the  error  in 

SA? 

.Unfamiliarity  / 

Lack  of  experience: 

Do  the  pilot(s)  cite  their 
unfamiliarity  or  lack  of 
experience  with  the  aircraft, 
flying,  crewmembers  which 
affected  the  SA  error? 

.Weather: 

Does  the  crewmember(s)  cite 
weather  as  a  factor  affecting  the 
error? 

.Fatigue: 

Does  the  crewmember(s)  cite 
fatigue  as  a  factor  affecting  the 
error? 

.Night  Conditions: 

Does  the  crewmember(s)  cite 
night  conditions  as  a  factor 
affecting  the  error? 

.Emotions: 

Does  the  crewmember(s)  cite 
being  upset  or  angry  as  a  factor 
affecting  the  error? 

.Distraction: 

Does  the  crewmember(s)  cite 
outside  sources  as  taking  them 
away  from  their  duties  to  fly  the 
aircraft? 
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7)  Was  high  workload  a  factor  affecting  the  SA  error?  Is  high  workload  cited 
by  the  reporters  as  a  factor  affecting  the  S A  error  or  is  it  apparent  from  the 
language  of  the  report? 

High  Workload:  Crewmembers  state  they  were  very  busy  during  flight 
operations  at  the  time  of  the  incident,  or  there  is  apparent  evidence  that  the 
incident  occurred  during  a  busy  portion  of  the  flight.  Extra  duties  above  normal 
operating  procedures  are  occupying  the  crewmember(s)  attention. 

Low  Workload:  Crewmembers  state  or  give  an  indication  that  the  workload  in 
the  cockpit  was  normal  and  not  adversely  affecting  the  flight. 

Unable  to  Determine:  No  apparent  evidence  is  provided  relating  that  flight 
operations  are  overly  busy 

High  Workload _  Low  Workload _  Unable  to  Determine _ 

8)  Was  time  pressure  a  factor  affecting  the  SA  error?  Is  time  pressure  cited  by 
the  reporters  as  a  factor  affecting  the  SA  error  or  is  it  apparent  from  the 
language  of  the  report? 

High  time  pressure:  Crewmember(s)  state  they  felt  time  pressure  affected  the 
flight  when  the  SA  error  occurred,  apparent  that  crewmember(s)  rushed  through 
procedures  to  meet  flight  demands. 

Low  time  pressure:  Crewmembers  state  or  indicate  that  time  pressure  was 
normal  when  the  incident  occurred  and  did  not  adversely  affect  the  flight 

Unable  to  Determine:  Crewmembers  do  not  state  they  felt  time  pressure 
affected  the  flight  or  no  apparent  evidence  is  provided  relating  that  time 
pressure  is  a  factor  affecting  the  incident 

High  Time  Pressure _  Low  Time  Pressure _  Unable  to  Determine 


9)  Indicate  the  phase  of  flight  in  which  the  incident  occurred.  Circle  the 
appropriate  phase. 

Preflight  Taxi-out  Takeoff  Climb  Cruise 


Descent 


Approach 


Landing 


Taxi-In 


Unable  to  Determine 
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